Bridging Simulations and Observations: GalSim & JWST Data Drive Deep Learning | HackerNoon

Table of Links

Abstract and 1 Introduction

2 Method

2.1. Overview and 2.2. Encoder-Decoder Architecture

2.3. Transformers for Image Restoration

2.4. Implementation Details

3 Data and 3.1. HST Dataset

3.2. GalSim Dataset

3.3. JWST Dataset

4 JWST Test Dataset Results and 4.1. PSNR and SSIM

4.2. Visual Inspection

4.3. Restoration of Morphological Parameters

4.4. Restoration of Photometric Parameters

5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images

5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images

6 Limitations

6.1. Degradation in Restoration Quality Due to High Noise Level

6.2. Point Source Recovery Test

6.3. Artifacts Due to Pixel Correlation

7 Conclusions and Acknowledgements

Appendix: A. Image restoration test with Blank Noise-Only Images

References

3.2. GalSim Dataset

For the generation of the pre-training dataset, we use GalSim based on the distributions of the Sersic parameters derived from the HST dataset to reproduce not only the marginalized distribution of each parameter but also the correlations between the parameters. We constructed the probability density functions (PDFs) in the five dimensional parameter space (axis ratio, Sersic index, flux, size, and noise) with the scipy.stats Gaussian kernel density estimator (KDE) module. Then, 100,000 new input parameter sets for GalSim were generated by resampling based on the derived PDFs.

The image size w varies in the original HST postagestamp dataset. However, the galaxy images in the training dataset need to have an equal size. We choose a size of 64×64 pixels since the choice provides a good balance between efficiency and image fidelity. A smaller image would truncate the profile too prematurely while a larger image would be dominated by background values.

The 100,000 noiseless galaxy images output by GalSim constitute the GT images in the pre-training dataset. Subsequently, these GT images were convolved with a 7×7 pixel Gaussian kernel whose width matches the HST PSF. Finally, Gaussian noise was added to align the rms noise level with that of the HST dataset. These degraded images comprise the LQ images.

Following the same procedure used for determining the best-fit Sersic parameters from the HST galaxy images, we measured the Sersic parameters from the LQ images. Figure 4 compares the Sersic parameter distributions between the HST and GalSim LQ images. One must understand that in principle the two distributions cannot align exactly because the input Sersic parameters used for the GalSim image generation are derived from the noisy HST images. Nevertheless, we find that overall the covariances and marginalized probability distributions are similar between the two datasets. Since the GalSim galaxy images are used to pre-train the model, which is subsequently finetuned by the JWST-based images, we do not think that the differences in detail matter.

3.3. JWST Dataset

We applied nearly the same procedures used for creating the HST dataset to sample the JWST image. The only difference is the criteria for the noise level. Since the JWST images prior to degrading serve as GT, the noise level must be considerably lower. We accepted an image only if its rms noise after the min-max normalization became less than 0.02.

One non-trivial issue is how to handle the difference in pixel scale between JWST and HST. NIRCAM’s native pixel scale is 0.03′′ (0.06′′) for the short (long) wavelength channel, which is different from the HST/ACS pixel scale 0.05′′. We considered resampling the JWST pixel scale to 0.05′′. However, the interpolation noise degraded the original quality non-negligibly. Therefore, we maintained the original scale. Therefore, our JWST galaxies are 40% larger or 20% smaller than the HST ones on average. However, as we demonstrate in §4, our HST image restoration is not significantly affected by this pixel scale inconsistency.

Another tricky issue is the JWST’s intrinsic PSF. Although JWST’s PSF is on average much smaller than that of HST, its long wavelength channel PSF width can be similar to that of the HST/ACS PSF. One may consider deconvolving JWST image with its own PSF as an attempt to further enhance its resolution and thus better represent the truth. Perhaps this is achievable with an approach similar to the current method explored here. However, this is beyond the scope of this study. In the current investigation, we use JWST images without deconvolution. Thus, the effective PSF of the LQ images that we describe below includes the convolutions by both JWST’s and HST’s PSFs.

We standardize the postage-stamp image size to 64 × 64 pixels. A smaller image is expanded by padding pixels to its four edges in such a way that both the noise and background levels match the original image. A larger image is trimmed to 64 ×64 pixels. We discarded the galaxy images whose profiles were prematurely truncated by the trimming procedure. We collected 113,485 postage-stamp galaxy images from the JWST NIRCam F115W, F200W, F277W, and F444W images from various fields publicly available in August 2023 the Mikulski Archive for Space Telescopes (MAST)[4]. The dataset serves as the GT component in our finetuning dataset.

The LQ counterpart is generated by convolving the JWST GT galaxy images with the HST PSF and adding Gaussian noises. Although the pixel scale is different from the HST image, we scaled the HST PSF size, assuming the 0.05′′ pixel scale. This is because, eventually, we desire to restore the HST images whose pixel scale is 0.05′′. The applied noise distribution follows that of the HST dataset. The ratios among the training, validation, and test datasets are 8:1:1.

[4] https://archive.stsci.edu/