Authors:
(1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea;
(2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA.
Table of Links
Abstract and 1 Introduction
2 Method
2.1. Overview and 2.2. Encoder-Decoder Architecture
2.3. Transformers for Image Restoration
2.4. Implementation Details
3 Data and 3.1. HST Dataset
3.2. GalSim Dataset
3.3. JWST Dataset
4 JWST Test Dataset Results and 4.1. PSNR and SSIM
4.2. Visual Inspection
4.3. Restoration of Morphological Parameters
4.4. Restoration of Photometric Parameters
5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images
5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images
6 Limitations
6.1. Degradation in Restoration Quality Due to High Noise Level
6.2. Point Source Recovery Test
6.3. Artifacts Due to Pixel Correlation
7 Conclusions and Acknowledgements
Appendix: A. Image restoration test with Blank Noise-Only Images
References
6. LIMITATIONS
6.1. Degradation in Restoration Quality Due to High Noise Level
Although our Transformer-based deep learning model provides state-of-the-art performance in both resolution enhancement and noise reduction for moderate noise levels, inevitably, proper restoration becomes impossible when the noise level exceeds a certain threshold.
Figure 11 shows four such examples. The top row illustrates a case, where the distinct spiral arm in the GT image is not restored. Upon scrutiny of the corresponding LQ image, we believe that the information loss due to noise is too significant to hint at the presence of the spiral arm. The second row shows a case, where the position angle of the GT image is not properly restored. Again, we suspect that the LQ image is too noisy to enable a proper inference of the position angle of the GT image. The third row is an example, where the ellipticity of the RS image is much rounder than that of the GT image. In the last row, we illustrate a case, where the deep learning model fails to unveil the two distinct peaks of the GT image.
Since the exact definition of restoration failure is a subjective matter, it is difficult to quote an exact threshold. Nevertheless, from visual inspections, we suggest that the failure frequency increases noticeably when the rms value (after min-max normalization) of the LQ image is larger than ∼0.1.
6.2. Point Source Recovery Test
In principle, the perfect deconvolution algorithm should restore a point source to a delta function, which is infinitely smaller than a pixel. In traditional deconvolution in the Fourier domain, it is a challenging task because the operation is numerically unstable. The resulting images often exhibit many ringing effects around a bright central peak.
Since we excluded stars from the training dataset, our deep learning model did not explicitly learn to deconvolve point source images. Thus, it is interesting to examine how well our deep learning model, trained with only galaxy images, restore point sources. We perform a point source recovery test as follows. First, we created 1,000 JWST-quality star GT images. Because we did not remove the JWST PSF from the training dataset, the star GT images should not resemble a delta functions but the JWST PSF. We implemented this by convolving a single pixel with a Gaussian whose kernel size matches the JWST PSF. Note that we randomize the positions of the stars within the central 24×24 pixels of the 64 × 64 postage-stamp images. Then, we created their LQ versions by further convolving the GT images with the HST PSF and adding noise. Finally, these LQ images are restored by our deep learning model. To investigate the systematic effect, we stack the 1,000 GT and RS images separately after aligning their centers.
Figure 12 shows that the difference is subtle when we compare the GT (left) and RS (middle) stacks visually. However, the residual image (right) illustrates that the PSF of the RS stack is systematically larger. Thus, we conclude that our deep learning model, trained solely with galaxy images, performs less than ideally for point sources.
6.3. Artifacts Due to Pixel Correlation
In our generation of LQ images, we assume that the noise is Gaussian. However, in real astronomical images, especially when we create deep images by stacking many dithered exposures, there exist significant interpixel noise correlations. We find that these inter-pixel noise correlations create non-negligible artifacts.
Figure 13 display some examples of these artifacts. The LQ images here are sampled from multi-epoch drizzled images. The presence of correlated noise is apparent
even from visual inspection. The RS images show that the correlated noise creates some low-surface brightness artifacts in the galaxy outskirts, which however are absent in the JWST images. Addressing this issue could involve strategies such as employing a different drizzling kernel for image stacking or leveraging more advanced deep learning algorithms. Exploring these solutions will be a key focus of our future work.
