Editing header of compressed image HDU without recompressing

I recently updated to astropy 7. Previously, using astropy 5, I had code that opened a fits file with mode “update” and replaced the header of a compressed image extension, but did not change the data. (The compressed HDU was created by running the command-line utility “fpack” on a single-HDU image, using all the defaults.) Under astropy 5, if I read that file again, the data I read was identical to what it had been before I did the update.

Now, when I read back the data from the updated file, the data is not identical to what it had been before. What this tells me is that astropy is reading the data and recompressing it to write it, and because it’s a lossy algorithm, it doesn’t write exactly the same compressed data.

Is there a way to tell astropy to change the header but not to uncompress/recompress the data? The parameter disable_image_compression of fits.open looked promising, but when I set that, I got the error KeyError: "Keyword 'TFIELDS' not found." (The documentation of disable_image_compression says it treats the image HDU as a binary table, so it seems to be really trying to make it a table.) I then added output_verify='ignore' to stop it from checking it was a table, but now got the error ValueError: cannot reshape array of size 1298880 into shape (4094,2046)

So is there a way to edit the header of a compressed image HDU without running the actual data through decompression and recompression? (There are two reasons to want this. First, performance: no need to spend the time decompressing and recompressing if I’m not looking at or changing the data. Second, and more importantly, I really don’t want to run data repeatedly through lossy compression.)

Astropy 7.0 includes some major changes for the compressed HDU class so we may have missed some use case like this one (if not correctly covered in the unit tests).
Could you please open a ticket in the astropy github, if possible with some example and traceback ?

1 Like