2022 article
Deep Residual Network with Multi-Image Attention for Imputing Under Clouds in Satellite Imagery
2022 26TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), pp. 643–649.
TL;DR:
A novel deep learning-based imputation technique for inferring spectral values under the clouds using nearby cloud-free satellite image observations is presented and it is demonstrated that the ECA method is consistently better than all other methods.
(via Semantic Scholar)
UN Sustainable Development Goal Categories
13. Climate Action
(Web of Science)
15. Life on Land
(Web of Science; OpenAlex)
Source: Web Of Science
Added: February 20, 2023
Earth observations from remote sensing imagery play an important role in many environmental applications ranging from natural resource (e.g., crops, forests) monitoring to man-made object (e.g., builds, factories) recognition. Most widely used optical remote sensing data however is often contaminated by clouds making it hard to identify the objects underneath. Fortunately, with the recent advances and increased operational satellites, the spatial and temporal density of image collections have significantly increased. In this paper, we present a novel deep learning-based imputation technique for inferring spectral values under the clouds using nearby cloud-free satellite image observations. The proposed deep learning architecture, extended contextual attention (ECA), exploits similar properties from the cloud-free areas to tackle clouds of different sizes occurring at arbitrary locations in the image. A contextual attention mechanism is incorporated to utilize the useful cloud-free information from multiple images. To maximize the imputation performance of the model on the cloudy patches instead of the entire image, a two-phase custom loss function is deployed to guide the model. To study the performance of our model, we trained our model on a benchmark Sentinel-2 dataset by superimposing real-world cloud patterns. Extensive experiments and comparisons against the state-of-the-art methods using pixel-wise and structural metrics show the improved performance of our model. Our experiments demonstrated that the ECA method is consistently better than all other methods, it is 28.4% better on MSE and 31.7% better on cloudy MSE as compared to the state-of-the-art EDSR network.