I am building my own processing pipeline following the Astropy CCDReductions guide and have some questions

Some details:
Telescope: WIYN 0.9 meter
Camera: cryo-cooled HDI 0.5º imager
Overscan: Present and useful

I am working on creating a script based on the Astropy CCDReductions guide here.

I am using Bias frames and Flat-fields but no Dark frames since it is a very cold detector. The issue I am having is that I am seeing some of my optical path obstructions in the processed data frames even after processing.

Some examples: (I can only post one image…)

Why do I see the ‘negative’ of that hair in the processed frame?

I noticed that the flat values were over 1 here, is that expected? I thought normalizing would make 1 the top?

Honestly, I think that maybe the wrong Flats are being used for the science images based on filter… Like a u-flat being applied to an r-image…

I will share my pipeline’s (rough-draft) code if anyone wants to take a look at it.

I confirmed that the right filtered flat images are being applied to my corresponding filtered science images.

Why do you see numbers above 1?

I think the relevant sentence in the guide is this “Typically the mean or median is scaled to 1.0 before combining so that when the science images are divided by the calibrated, combined flats, the science image values do not change too much.”. In other words, you can scale it almost in any way you like. If you put the median to 1, then half the pixels will have values above one and half the pixels will have values below. What the flat is for is to correct relative differences in your image (e.g. if one pixel is less sensitive than a neighbor or is always one third of the light is lost due to obscuration for the pixels on the edge. You will alter flux-calibrate your data based on some astronomical object to put it on an absolute scale.

Why do you still see stuff the flat-field is supposed to take out?

It’s hard for me to know if that’s a problem. No correction is ever perfect in reality, so think about your science. Do you need to know it any better than you have it now? There could be a number of reasons for that, e.g. dome flats can be different from sky because the source is at a finite distance (in fact, the dome it typically pretty close to the telescope!), twilight flats can be illuminated unevenly depending on the atmosphere, if you have broad band filters and the source (dome, twilight) has a different spectrum than your target… For that kind of thing you should talk to other users of the same telescope and instrument. It might not be Python specific.

Of course, it could also be a programming error (e.g. you typed “*” instead of “/” at some point so any other stupid typo like that). For that, I can only recommend to look at every intermediate product and check where it starts to look odd. But it seems you’ve done that already, so maybe that’s not a likely solution.