Opening image file using astropy gives me wrong numbers

Astropy
,
1

Hi. I’m trying to open some of the model maps of Fermi LAT data made by fermitools, and there’s some problem of opening it.

When i try to open the image file(fits) made by gtmodel outtype=ccube(it’s fermitools option), the image values won’t give me the proper ones every time i open the file using astropy.

Problem is, whenever i open it with fits viewer, the numbers are correct, but using astropy, numbers are quite weird.

For example, if i open the model file using fits viewer, the number for each pixels are

0.798225 0.80624 0.81207

0.825374 0.826063 0.827172

0.860138 0.865281 0.86977

for the first 3x3 matrix.

But opening it using astropy, using fits.open, then i get numbers for corresponding array elements to be

array([[1061968001, 1062102462, 1062200277],
[1062774961, 1062906993, 1063001653],
[1063546681, 1063657674, 1063732791]], dtype=‘>i4’)

I don’t see any scaling laws between those two groups of numbers, and i’m just guessing that it might be the problem with one of the header components. I’m uploading my header tables for the gtmodel file.

SIMPLE = T / File conforms to NOST standard
BITPIX = -32 / Bits per pixel
NAXIS = 3 / No data is associated with this header
NAXIS1 = 200 / Length of data axis 1
NAXIS2 = 200 / Length of data axis 2
NAXIS3 = 24 / Length of data axis 3
EXTEND = T / Extensions may be present
COMMENT FITS (Flexible Image Transport System) format is defined in ‘Astronomy
COMMENT and Astrophysics’, volume 376, page 359; bibcode: 2001A&A…376…359H
CTYPE1 = ‘GLON-AIT’ / RA—%%%, %%% is the projection, e.g., AIT
CRPIX1 = 100.5 / Reference pixel
CRVAL1 = 359.947801 / RA at the reference pixel
CDELT1 = -0.2 / X-axis incr per pixel at ref pixel (deg)
CUNIT1 = 'deg ’ / Physical unit of X-axis
CTYPE2 = ‘GLAT-AIT’ / DEC—%%%, %%% is the projection, e.g., AIT
CRPIX2 = 100.5 / Reference pixel
CRVAL2 = -0.036992 / DEC at the reference pixel
CDELT2 = 0.2 / Y-axis incr per pixel at ref pixel (deg)
CUNIT2 = 'deg ’ / Physical unit of Y-axis
CTYPE3 = ‘photon energy’ / log_MeV
CRPIX3 = 1. / Reference pixel
CRVAL3 = 300. / energy at the reference pixel
CDELT3 = 50. / z-axis logrithmic incr per pixel
CUNIT3 = 'log_MeV ’ / Physical unit of Y-axis
CROTA2 = 0. / Image rotation (deg)
DATE = ‘2023-11-22T09:55:19’ /
TELESCOP= 'GLAST ’ / Name of telescope generating data
INSTRUME= 'LAT ’ / Name of instrument generating data
DATE-OBS= ’ ’ / Start Date and Time of the observation (UTC)
DATE-END= ’ ’ / End Date and Time of the observation (UTC)
EQUINOX = 2000. / Equinox of RA & DEC specifications
CREATOR = 'gtmodel ’ / Software and version creating file
HISTORY $Id: LatCountsMapTemplate,v 1.2 2004/09/24 03:54:20 jc
HISTORY hiang E
CHECKSUM= ‘0000000000000000’ / HDU checksum updated 2023-11-22T00:55:19
DATASUM = ‘3894447489’ / data unit checksum updated 2023-11-22T00:55:19
SIMPLE = T / File conforms to NOST standard
BITPIX = 32 / Bits per pixel
EXTEND = T / Extensions may be present
COMMENT FITS (Flexible Image Transport System) format is defined in ‘Astronomy
COMMENT and Astrophysics’, volume 376, page 359; bibcode: 2001A&A…376…359H
CTYPE1 = ‘GLON-AIT’ / RA—%%%, %%% is the projection, e.g., AIT
CRPIX1 = 100.5 / Reference pixel
CRVAL1 = 359.947801 / RA at the reference pixel
CDELT1 = -0.2 / X-axis incr per pixel at ref pixel (deg)
CUNIT1 = 'deg ’ / Physical unit of X-axis
CTYPE2 = ‘GLAT-AIT’ / DEC—%%%, %%% is the projection, e.g., AIT
CRPIX2 = 100.5 / Reference pixel
CRVAL2 = -0.036992 / DEC at the reference pixel
CDELT2 = 0.2 / Y-axis incr per pixel at ref pixel (deg)
CUNIT2 = 'deg ’ / Physical unit of Y-axis
CTYPE3 = ‘photon energy’ / log_MeV
CRPIX3 = 1. / Reference pixel
CRVAL3 = 300. / energy at the reference pixel
CDELT3 = 50. / z-axis logrithmic incr per pixel
CUNIT3 = 'log_MeV ’ / Physical unit of Y-axis
CROTA2 = 0. / Image rotation (deg)
DATE = ‘2023-11-21T11:55:15’ /
TELESCOP= 'GLAST ’ / Name of telescope generating data
INSTRUME= 'LAT ’ / Name of instrument generating data
DATE-OBS= ’ ’ / Start Date and Time of the observation (UTC)
DATE-END= ’ ’ / End Date and Time of the observation (UTC)
EQUINOX = 2000. / Equinox of RA & DEC specifications
CREATOR = ‘gtsrcmaps’ / Software and version creating file
HISTORY $Id: LatCountsMapTemplate,v 1.2 2004/09/24 03:54:20 jc
HISTORY hiang E
CHECKSUM= ‘JkF8Mj95JjC5Jj95’ / HDU checksum updated 2023-11-22T00:55:19
DATASUM = ‘1817100391’ / data unit checksum updated 2023-11-21T02:55:15
DSTYP1 = ‘POS(RA,DEC)’
DSUNI1 = 'deg ’
DSVAL1 = ‘CIRCLE(266.41,-29,50)’
DSTYP2 = 'TIME ’
DSUNI2 = 's ’
DSVAL2 = 'TABLE ’
DSREF2 = ':GTI ’
DSTYP3 = ‘BIT_MASK(EVENT_TYPE,3,P8R3)’
DSUNI3 = ‘DIMENSIONLESS’
DSVAL3 = '1:1 ’
DSTYP4 = 'ENERGY ’
DSUNI4 = 'MeV ’
DSVAL4 = ‘300:500000’
DSTYP5 = ‘ZENITH_ANGLE’
DSUNI5 = 'deg ’
DSVAL5 = '0:90 ’
DSTYP6 = ‘IRF_VERSION’
DSUNI6 = ‘DIMENSIONLESS’
DSVAL6 = ‘P8R3_ULTRACLEANVETO_V3’
DSTYP7 = ‘BIT_MASK(EVENT_CLASS,1024,P8R3)’
DSUNI7 = ‘DIMENSIONLESS’
DSVAL7 = '1:1 ’
NDSKEYS = 7

This is quite of problem because i’m having some of the difficulties to compare model maps and counts maps. Although i can scrap some of the model map informations using fits viewer, since it gives me the correct numbers, but i’m wondering what would be the problem opening this file with astropy. Thank you.

2

Hi,
there is no scaling in any of the headers that I can see - but for some of the keywords that you posted, there is also no information on the shape of number of pixels. Can you post the code that you use in astropy to read the data?

Also, can you post the output of hdu.info() which should list the shape of all components of the file (see FITS File Handling (astropy.io.fits) — Astropy v6.1.dev115+g1aa3e6a5 for an example).

3

Thanks for the attention.

The code i used to get the image data is :

from astropy.io import fits

image_data=fits.open(‘Fermi_model_map.fits’)

print(image_data[0].data[0]) #I have 24 images for 24 energy bins. image_data[0].data[0] gives me the image of the first energy bin.

Result of this is :

[[1061968001 1062102462 1062200277 … 1062203266 1062258944 1062327905]
[1062774961 1062906993 1063001653 … 1062965008 1063007412 1063059357]
[1063546681 1063657674 1063732791 … 1063610104 1063639280 1063677595]
…
[1074676190 1074937704 1075226140 … 1069018414 1069313651 1069539082]
[1074558357 1074805685 1075075510 … 1068997227 1069244655 1069450765]
[1074443634 1074675231 1074925060 … 1069047308 1069253473 1069438407]]

For the dimension of my data,

np.shape(image_data[0]) → (24, 200, 200)
np.shape(image_data[0].data) → (24, 200, 200)
np.shape(image_data[0].data[0]) → (200, 200)

and image_data.info() gives me

No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 91 (200, 200, 24) int32
1 GTI 1 BinTableHDU 33 11259R x 2C [D, D]
2 EBOUNDS 1 BinTableHDU 38 24R x 3C [I, 1E, 1E]

The PRIMARY table gives me the actual pixel values, and GTI table gives me informations about space craft maneuver(where is was, and where it was pointing at), and EBOUNDS table gives me the definitions of energy binning.

I’m thinking about the Format of the 0-th table(PRIMARY) might be the problem of image data. Because my model map’s actual pixel data are between 0 to around 5, in decimal, but the format of the table read by the astropy is int32.

4

I think i found a reason, which is i had two header named ‘BITPIX’, one is -32, that indicates float-32, and other one is 32, indicates int32. Conflicts between these two data type made floating data type to convert into int type in very inappropriate way. But the problem is, i couldn’t set one of two ‘BITPIX’ header into -32, and whenever i try to set my ‘BITPIX’ header into -32, only one of them changed, and other one remained as 32. I forced to change the value by deleting header itself using fits.delvat(‘file_name’, ‘BITPIX’) and try to change the number of 32 into -32 with fits viewer.

Weird thing is, whenever i try to change the value for ‘BITPIX’ with fits.setvar(‘file_name’, ‘BITPIX’, value=-32, ext=0), it doesn’t change each two ‘BITPIX’ values, it only changes the first ‘BITPIX’ value, so that second ‘BITPIX’ just remain as 32.

Now i am trying to figure out why astropy is not setting my header properly, and how to set my header ‘BITPIX’ value as -32 for both header components.

5

Do you mean that you have two keywords names BITPIX in the same extension? In FITS, header keys are supposed to be unique (except for HISTORY and COMMENT). So, if your header has the same keyword twice, it’s not actually a valid fits file. You can get to the second keyword by number if you need to (FITS Headers — Astropy v5.3.3) but there really should be only one.

6

Yes, that is what i am saying. My PRIMARY extension has two keywords with same name but different value. For some reason, fits.delval(‘file_name’, keyword=‘BITPIX’) won’t delete both of the keyword but only first keyword. Same for the fits.setval(‘file_name’, keyword=‘BITPIX’, value=-32). So i decided to just apply fits.delval(‘file_name’, keyword=‘BITPIX’) to remove one of the keywords, and set remaining one’s value as -32 by my hands with VIM. This solved the issue, but still i have no idea why the fits.setval and fits.delval won’t delete unwanted keyword named ‘BITPIX’ has has value of 32. It only removes another ‘BITPIX’ keyword with value -32.

7

fit.delval will remove one and only one keyword. If a keyword is in the head twice, you have to call it twice.

However, a much better fix would be to ask the author of the file (or the author of the program that wrote the file) to fix their program that that there is only one BITPIX keyword in the first place. Having two with different values in the same header makes no sense.