Flats not working?

Ronald Clanton

Adam,

I'm back again with another issue.  Over the last couple of sessions, I'm having a problem with WBPP in processing my images.  It doesn't appear that it is applying my flats to the image... the gradient is quite large (although I can process that out) and there are bad pixels showing (particularly in the right/top quadrant).  Those bad pixels are particularly troubling.  My flats match up perfectly with those pixels, but don't seem to be calibrated out (see attached).

I've reset to factory settings for WBPP, added lights, flats, darks and bias frames.  The calibration diagram looks correct.  Only change is to reference the cosmetic correction icon and designate an output directory.

Here is a link to the master files.

https://www.dropbox.com/s/u67l4o2hwioaoo2/M109Archive.zip?dl=0

Since this is a new problem (only in the last few days), I've racked my brain on what's changed.  I'm imaging with a near full moon, so my expectations are low... but it doesn't explain the bad pixel happening.  I hope you can help!

Thanks,

Ron

Ronald Clanton

Just as a follow-up...

I ran everything through Astro Pixel Processor (I used before PI) and they calibrated fine.  So... there must be something I'm doing wrong in PI.

Ron

BlockHead

Hi Ronald,

You need to follow the procedure of FastTrack Training until you become proficient in identifying the hangups.

I think this image is the integration of non-aligned frames. I do not see any stars ... and that smudge is the rejected galaxy? You need to blink the calibrated frames... then blink the registered frames... etc. I think these frames are not registered at all..but you need to make certain the calibration is good as well.

Please look at every frame... and then you show no the problem frames.. but you should show all the data that WORKED at each step. You should basically say... "it can't be this issue... look the calibrated frame is fine" "It can't be that the frames are not aligned... " etc etc. By showing all of the things is cannot be... THEN we can address the issue. 

(usually in the case of non-aligned frames... not using cosmetic correction is to blame)

-the Blockhead 

Ronald Clanton

Thanks!

I did blink all the light frames and they have stars in them.  I will follow your advice and track them through the stages to diagnose the problem.  It's just weird that this problem has happened only in the last few nights when I've been using PI/WBPP for almost a year.

Oh well...

Thanks,

Ron

Ronald Clanton

Okay Adam.  I followed your advice and have discovered the issue.  None of those defects were eliminated in calibration or cosmetic correction.  Based on prior integration runs that worked, they are normally eliminated in the integration stage... based on a good registration.

As you noticed there are only a few real stars in the image, so it chose to register/align based on the defects... which caused the galaxy to blur.  This is also true on the other images I've had issues.  Lately, I've been imaging areas that have few stars.

So... I'm trying to figure out how to adjust either the calibration step or cosmetic correction to eliminate the problem before it gets to registration.  It seems that calibration would be the best place, but I have tried/tested several parameters without success.

Am I on the right path? or should I just adjust the image integration parameters to keep it from "seeing" those defects?

Thanks,

Ron

BlockHead

Ron,

Please see the bottom of FastTrack Training:

https://www.adamblockstudios.com/categories/pixinsight-fasttrack-training-beginners

You will see I have a video that anticipates this common issue. 

"

• Step 7: Learn the *hard* lesson now! (as mentioned in the training lesson) StarAlignment, Debayering and Hot Pixels"

The problem is that if you do not eliminate the hot pixels, they might mess up the registration process... especially in a field like M109 where there are very few stars. 

Yes, rejection should take care of them... but the issue is happening *earlier* than ImageIntegration. The fact that you have hot pixels could also be a symptom your dark frame is not matching your light frame well..but this depends on the sensor. 

In my videos I discuss the different method of hot pixel correction you can use. Auto is typically fine...but using the dark frame method is even more brute force. It is a very good idea to learn all of these things.

-the Blockhead 

Ronald Clanton

Thanks Adam!

I don't know how I missed the video when I went through FastTrack... but it was very helpful.  It helped me analyze the root cause of my problem.  It certainly validates my decision to opt-in for the PixInsight Horizons plan last month!

However, what I uncovered still doesn't make sense to me.  I've attached a screen capture that shows a raw light capture on the left and a raw flat capture on the right.

First, you'll notice that the flat indicates a bright group of pixels surrounded by black pixels.  This looks like more than a simple hot pixel... probably a camera defect?  I looked back through my previous flats and this only started occurring in the last couple of weeks... and then suddenly in about 50 places.  I'll probably need to contact ZWO to get their thoughts on the problem.

Second, what I don't understand is that the bad pixels don't appear at all in the 300 second raw light capture or matching dark frame, so during calibration it corrects what's not a problem... and I get small donuts in the calibrated frames!  I don't understand how this is possible.  Have you ever seen this occurring?

Thanks,

Ron

BlockHead

Hi Ron,

You are absolutely right to question this. Yes, you would very much expect those pixel things you see in your flats to appear in your data. One difference, I suspect, between your flats and data is that your exposure times for the flats are short. I wonder if your camera is acting weird for the short exposures. In order for the flats to work- they need to characterize the sensor in exactly the same way it is behaving when taking light frames. These flats would certainly not do that with respect to these weird pixels.

I do not know the cause, and I have not seen this before. 

But..I will say good job in looking closely at the data... now you have a good place to begin figuring it out.

-the Blockhead