PSF Photometry That Doesn't Gobble Up All Available Memory

Astropy
1

Hello! I’m trying to run PSF photometry on a set of crowded fields following this (potentially outdated) stsci workbook. I’m following the notebook, which uses the photutils procedure BasicPSFPhotometry, and while I don’t have to generate the field myself (I am using some simulated imaging), the results on an individual image look great. The issue is, it takes ~2 hours to run on ~5000 stars. My first question is how I might go about speeding this up. I made an attempt to use multiprocessing to run my script in parallel on a set of cores, but each time I attempted to do this 4x, after a while the processes quit because it ate up all of the available memory (and there’s 64 GB, so this is a hefty ask!) I recognize that PSF photometry is not super straightforward, but it would be helpful to know if there are things I can change such that I can run things on a larger image data set. Thank you!

2

It’s been a while since I looked at the PSF code, but yeah, a large number of stars takes long and east memory like crazy. I’m going to suggest a few general strategies, but I don’t know if any one of those is appropriate for your specific case:

  • fix as many parameters as you can (e.g. measure the width once and then fix it for all. Even if that number is slightly off, the total flux will usually still be fitted well.)
  • Do the PSF fit with fixed X,Y determined from the source finding (PSF Photometry (photutils.psf) — photutils 1.3.0)
  • Chop your image in bits, e.g. into four (slightly overlapping to deal with stars that are otherwise on the edge) quarters. Running 4 small images can be much faster than one big one.
  • re-sample your image. Sometimes you can bin up (e.g. binning by a factor of 2 reduces size by a factor of 4) and still sample the PSF well - depends on your data if that works.
  • Do your fit in parts. E.g. get flux with fixed X,Y, then re-fit flux, X, Y, sigma all at the same time, but when you do that use the previous results to start the fit so that it’s starts pretty close to the best-fit values. The closer you start, the faster the fit.
3

These are great suggestions! The key for me was updating to astropy 5.0.0, which didn’t cause the PSF fitting to fill all available memory and crash. and then another important thing, which was actually changing the sampling and size of the PSF images that I was using for the fit. By default webbPSF produces very oversampled PSFs, and by changing the way that these PSFs were made, I went from ~2 hours to ~10 minutes, with very little change in the results. For me, I couldn’t fix the X/Y values in the PSF subtraction because this was part of what this procedure was for - to measure more precise centroids for the stars in the image. Thank you!