Space Selfies Level Up-AI Learns from the Best in the Universe | HackerNoon

Table of Links

Abstract and 1 Introduction

2 Method

2.1. Overview and 2.2. Encoder-Decoder Architecture

2.3. Transformers for Image Restoration

2.4. Implementation Details

3 Data and 3.1. HST Dataset

3.2. GalSim Dataset

3.3. JWST Dataset

4 JWST Test Dataset Results and 4.1. PSNR and SSIM

4.2. Visual Inspection

4.3. Restoration of Morphological Parameters

4.4. Restoration of Photometric Parameters

5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images

5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images

6 Limitations

6.1. Degradation in Restoration Quality Due to High Noise Level

6.2. Point Source Recovery Test

6.3. Artifacts Due to Pixel Correlation

7 Conclusions and Acknowledgements

Appendix: A. Image restoration test with Blank Noise-Only Images

References

3. DATA

The goal of the current study is to construct a Transformer model that enhances the quality of HST images to that of the JWST. Thus, our training data set consists of pairs of JWST-quality ground truth (GT) images and their degraded, low-quality (LQ) versions. We employ a transfer-learning approach by first pre-training our model with GT-LQ pairs of simplified galaxy images and finetuning the pre-trained model with GT-LQ pairs of realistic galaxy images. Therefore, we prepare two sets of training datasets.

The GT image of the pre-training dataset is generated with GalSim (Rowe et al. 2015) while its LQ counterpart is obtained by simply degrading the resolution of the GT image with the HST point spread function (PSF) and adding Gaussian noise. The input parameters of GalSim should not be arbitrary but must be chosen carefully to represent the structural features of real galaxies. This is achieved by harvesting real galaxy images from the HST archive and measuring their properties.

The GT image of the finetuning dataset is obtained by collecting galaxy images from the JWST archive. Following the same procedure in the pre-training dataset generation, we create their LQ counterparts. Below we describe the HST, GalSim, and JWST datasets in detail.

3.1. HST Dataset

We collect multi-band galaxy images from the following HST/ACS deep fields: the Hubble Ultra Deep Field 12 (HUDF12), HUDF-Parallel 2 (HUDFP2)[2] (Ellis et al. 2013; Koekemoer et al. 2013), and the Great Observatories Origins Deep Survey North and South (GOODSN&S)[3] (Illingworth et al. 2016; Whitaker et al. 2019) fields. The F606W, F775W, and F814W filters are used from HUDF12. For HUDFP2, only F814W is selected. We employ F606W and F814W from GOODS-North&South.

SExtractor (Bertin & Arnouts 1996) was used to detect sources by searching for five connected pixels 1.5 times above the rms noise level. We cropped a w × w pixel postage-stamp image centered on each source. The image size w is set to 2[4a+5], where a is the semi-major axis output by SExtractor and the operation [x] represents the greatest integer not exceeding x. We discard the source if

1. its image size w is greater than 128 pixels,

2. either its semi-minor axis or half-light radius is smaller than the FWHM of HST/ACS PSF,

3. its S/N value is less than 20,

4. the maximum value of its neighboring sources is larger than 0.8 times its peak value, or

5. the gradient within the central 10 × 10 pixels is sharper than the HST PSF.

The first and second criteria eliminate excessively large or small sources. The third condition removes excessively faint sources. The fourth condition excludes the cases where similarly bright galaxies are present within the postage stamp image. The last condition is necessary because the HST automatic cosmic ray detection/removal pipeline often fails near the bright peaks of stars or galaxies. In addition to these criteria, we explicitly identified and removed stars using their distinct locus in the size-magnitude diagram. We obtain a total of 66,092 postage-stamp images. Figure 3 shows the distribution of the HST postage-stamp images prior to the w = 128 cut.

As mentioned above, our pre-training dataset is created by GalSim using the characteristics derived from our HST dataset. We choose to characterize HST galaxies by their best-fit elliptical Sersic parameters and fluxes. The circular Sersic profile is defined as

where n represents the Sersic index, R is the radius, R50 is the half-light radius, and I(R50) is the surface brightness at R50. The elliptical Sersic profile is implemented by replacing R with the following:

where a and b denote the semi-major and semi-minor axes, respectively.

Because of the rapid change in I(R) near the center (R = 0), evaluating I(R) only at discrete positions (i.e., at the center of each pixel) leads to a significant discrepancy between the rendered and real galaxy images. To mitigate this, we first oversampled I(R) with ten times higher resolution in the central region (R < 5 pixel) and then applied 10 × 10 nearest neighbor-averaging to render the galaxy. This Sersic galaxy image is convolved with the HST PSF and then fit to the HST galaxy image. Fluxes are measured by summing up the backgroundsubtracted pixel values from the HST galaxy images.

[2] https://archive.stsci.edu/prepds/hudf12/

[3] https://archive.stsci.edu/hlsps/hlf/