›

Flats again...

Wesley Mitchell
in PixInsight
These images were taken from a remote rig in Texas.
I have to use sky flats to capture flat images.

It looks to me like the flats are overcorrecting by a lot. I have a zip file with a raw light, a single flat, the master flat, and a raw calibrated light. I can't see any of these rings in the raw light, no matter how I stretch it.

So I must be doing something wrong. I'm doing the processing inside WBPP. Can you comment? Files located here: 

Thanks
Wes

Comments

  • BlockHeadBlockHead
    Sorry... before I look .. you did not provide the DARKS . This is the critical piece of evidence.
    You need to demonstrate you are calibrating the flat correctly and that what you are calibrating with (the dark) is valid.

    -the Blockhead
  • Wesley Mitchell
    Hi Adam

    I added a master bias and master dark. The new link is this.
    Thanks

  • BlockHeadBlockHead
    Sorry I forgot to mention this.. you also need to set that link you created/copied to have permissions for "anyone that has the link"  . I do not have permission currently to see your files. You need to adjust the permissions for that link. 

    I demonstrate this in a video:

    -the Blockhead
  • Wesley Mitchell
    Done
  • BlockHeadBlockHead
    edited April 22
    Wes,

    Please look at the values of your masterdark compared to your masterbias.
    They are incompatible. Your masterdark has values of 0.0000xxxx  whereas your masterbias (which should be equal to or maybe a tiny bit less than this) have values of 0.015xxx .

    I have many videos where I explain the first step to solving calibration problems is to look at the values of your darks, biases and flats. 

    So there are quite a few problems..and I don't know where to begin. So I am going to list some observations.

    1. Your raw light Blue frame has the wrong metadata (in the FITs Header). It is labeled as a master flat with a tiny exposure time. It appears...and I am totally guessing here..but you were messing around with the header information (copying it or something). I do not know how you did this. 
    2. Because of the above- if you used WBPP in this manner you might be applying real light frames as MasterFlats!! 
    3. Master files do not carry all of the information.. you always need to provide examples of RAW darks and biases...not just the masters.
    4. Your MasterDark looks bad. It has ZEROS in it. This means you were likely exposing your dark frame at a low GAIN setting (or Offset) compared to your Bias frame. Your dark frame setting is no good. 
    5. Your GAIN and OFFSET settings must be the same for all frames...I do not believe this is the case.

    The current set of data is no good. You cannot fix this in processing... you need to get proper calibration files that match the lights.

    -the Blockhead
  • Wesley Mitchell
    Hi
    I'm looking at a second, older set of data that is more consistent. Everything is taken at gain100, offset 256. These were probably the defaults that came up from the driver when I first connected the camera.

    The master dark and master bias are pretty consistent. Random spot checks reveal pixel values around 0.015, median 0.0157, the md is very slightly larger than the master bias.

    However, the raw light frames have median 0.0034. So I would guess my darks are way too light?

    Help me understand the math here. If I take the light frame and subtract the master dark, a lot of those values are negative. Do they get truncated to 0? Then what happens when you divide by the pixel value of the masterFlat/med(masterFlat)? What's the final value for that pixel in the calibrated light frame?

    Thanks
    Wes
  • Wesley Mitchell
    Could you point me at one of your videos where you do analysis of the calibration frames by looking at the statistics?
    Thanks

  • BlockHeadBlockHead
    I think just looking at the values as you are doing is good enough. I don't think image statistics will necessarily provide more information for these purposes. 

    It is enough to know your calibration frames and light frames do not match. The calibration data does not match the electronic signature of the sensor (for many possible reasons, I named a few). 

    When you take raw light frames the electronic signature plus the sky brightness will almost always leave you with positive values after proper subtraction of darks/biases. You are correct that there are no negative numbers in our images. Oversubtraction results in ambiguous (truncated)  zeros which is a loss of information. 
    This is why for narrowband images a pedestal is required during image calibration. NB images have virtually no significant sky values. 

    I suspect for the flats- zeros become ones or something after calibration. I don't know. But that is academic how non-useful values are handled. 

    The video I would point to is just a long version of what you have already found by looking at the values:
     Look a the problem solving section.
    The videos about the CMOS darks and flats is the reference (3 videos). There are other examples in that list as well.

    -the Blockhead
  • Wesley Mitchell
    edited May 7
    Hi, Adam

    Back again, this time with a different dataset. Everything taken at same gain offset, and temperature (approx).
    I examined the dark frames, bias frames, and flat frames fairly closely. One thing I did notice is that the calibrated flat frame seemed too dark: the values on the flat are right around 0.5, the values on the 2-second master dark and master bias are both around 0.016, but the values on the calibrated flats are .37-.38. Shouldn't they be higher, in the neighborhood of .48?

    The calibrated lights are overcorrected. I mean that the vignette is lighter than the background, and so are the rings for the dust spots.

    I tried "tuning" the flats per one of your troubleshooting videos. Only when I essentially clipped the flats at the high end, did I see any improvement.

    What causes this overcorrection? 

    Is it because I took all these exposures with such a high offset? With a different setup, different camera, but same chip, the 1sec darks are around 0.003, same gain, but offset of only 20.

    I still struggle to understand the math. For those pixels that are overcorrected, the value in the flat is a bit less than median, so dividing by f/med(f) is dividing by a number less than one, which makes the result larger (brighter). However, these values are too large (too bright). What I don't understand is, what is the effect of subtracting off the bias from this flat? If the bias is too large, does that make the ratio smaller, which makes the correction larger?

    Here's a link to a zip file; I've got a few each of raw flat, raw 180s dark, calibrated flat, raw light, calibrated light, and the masters

    Thanks for your help
    Wes
Sign In or Register to comment.