Flat field errors

Ray Hansen

Hi Adam,

Many months ago, I read one of your posts that talked about a method (pixel math maybe?) to address the flat field overcorrection as illustrated in

the attached. Of course, I didn't make notes on it, and now can't find your instructions. This is something I commonly encounter, and have always fixed in photoshop, but I would like to be able to get it done quickly in PI.  When I try to fix it with DBE, I don't usually have much success. If you (or anyone else) can recall where this lesson is located, I would appreciate your help.

best regards,

Ray

BlockHead

Ray,

Fundamentally the overcorrection is telling you there is an issue with the darks/biases that are being used to calibrate the flat field images. Even worse, is that the flats are perhaps not be calibrated at all. Or there could be a configuration issue. In the video you are referring to I demonstrate that there is a pedestal value (either positive or negative depending on the error) causes these issues. You can diagnostically demonstrate this by adding (or subtracting) a small value to the flats...and then they work. But the idea is to point you in the direction of fixing the flat acquisition or darks or whatever...not to use this in general. 

There were three parts... but this is the last part:

https://www.adamblockstudios.com/articles/cmos-darks-and-flat-darks-part-3

You can find it on this page with the other parts under "Problem Solving Techniques."

https://www.adamblockstudios.com/categories/FundamentalTechniques

Searching for "flats" at the top of the site does bring up this video as a first one.

-the Blockhead

Ray Hansen

I created new darks, bias frames, and flat darks, and processed everything in WBPP.  I don't seem to be getting any improvement. I can do the pixel math tuning, but don't get acceptable results until I'm at +0.7 or more. Does that still sound reasonable?

Ray Hansen

Here's a clue. The attached was produced with WBPP, with no flats, darks only. 

BlockHead

This is unreasonable. It means the flats are no being calibrated correctly.

You need to make certain you are not changing gain settings or software in any way between types of data. All has to be the same. You need to check your configuration in WBPP is correct.

Finally, you need to upload the data to place I can access. 5 frames of each file type would be sufficient. If you do upload to a cloud server- you need to specify anyone with the link can access.

-the Blockhead

BlockHead

Ray,

One of the things that is very subtle that people do constantly (drives me a little mad) is take calibration data with different software programs. This can definitely cause issues. 

You did not do that...

But what you DID do was UPGRADE your software/driver..and guess what, with certain companies...this matters. I can see in the FITS header your flats and darks were taken with different versions. 

New calibration data will not solve this and fix the older Flats! 

(please read the above carefully!)

I have some suggestions...

1. I have the same camera as you...and it has a cooler... is this the lowest temp you can achieve? (0 C?) You are operating at a high temperature..and you are not matching temperatures (this is not optimal either)

2. You should a complete set of calibration data and lights with your current software holding everything constant (gain) and see if things work. Because I found something I think you need to do more proofing before I am convinced it is something else.

3. Producing an image with no calibration at all isn't informative. Calibrated data will always be better at some level.

So I think my last message was on target "you need to make certain you are not changing software in any way between data types... " 

Please do a complete set of new data...everything contemporaneous!

-the Blockhead

Ray Hansen

Hi Adam, 

1. I don't know if you work on the mountain in Tucson, but in the summers in Scottsdale that camera is groaning when it gets to about -2c. Until the monsoon starts, I can use lower temps for sequences after midnight. During monsoon though, the temp here never goes below about 85. Of course, for the rest of the year its a different story. I may try a different fan when I have time.

2. I dropped back a couple Sharpcap releases, and reshot darks and biases. Amazing. I would never have believed that would be the solve, but all is fine now. Many thanks to you. It got me thinking though, is PI backward compatible?

3. No argument, but I think I'll ask Robin if he can suggest why there's a difference between the two releases.

best regards,

RH

BlockHead

Ray,

An interesting challenge would be to present the same problem as you did with me and see if anyone else can solve it (as quickly). Maybe pass on the word to Robin that I am genuine?

PI is not always backward compatible in certain areas. Not so much in calibration... but other things like astrometry for example. In fact, there are some changes in the next release that are definitely not.

-the Blockhead