Multicolor Gradient M33

Kenneth Toliver

Here is a link to my integrated image:

Combined.xisf

Shot with an iOptron 10" RC Truss and TS Optics .8x reducer at 1625mmFL f/6.4. Stack of 679, 30 second subs calibrated with bias, darks, and flats. There were very high thin clouds those few nights and I believe that may be the contributing factor.

I cannot seem to deal with the gradients and background. I've tried ABE, DBE, GraXpert, and just end up with a mess, especially after trying to color calibrate. Any recommendations would be appreciated.

BlockHead

Yeah... something does not seem right about the image itself.

Maybe the clouds... but the image seems crushed already. There is a very large gradient that I would have thought flat fielding would have helped with. I think you need to look back at the calibrated data and make absolutely certain it makes sense... especially that there is no oversubtraction... (bad darks).

-the Blockhead

Kenneth Toliver

Yeah, I've looked at it several times, but I'll give it another go. It is the first time I've used true sky flats so that might be  factor. I wonder if using a pedestal during calibration of the flats makes sense? 

And please tell me what you are seeing to consider the image "crushed".

BlockHead

It just seems like the values are tiny. The stretch is so large that there is a quantization of the data.

That is the reason your gradient remove doesn't work... not one of the methods would handle this situation.

I don't understand how you achieve this... something about the rejection or the original calibration...I just do not know.

I would have to download your set and process it... but I just can't do that for everyone. So we need to collaborate. You will need to show many screenshots of calibrated data.. showing the values of the calibrated lights, the values of darks and biases. I do not think the sky flats are an issue unless they really are bad flats (possible if there is excess light getting in there).

I do think my observation that you masterlight is not flatted is important. So yes, looking at the flats is a good place to investigate on top of everything else. Assume nothing is correct... but then SHOW ME everything has to be correct with screenshots. That is how to proceed. At some point you or I will discover the problem when you try to prove everything is correct.

-the Blockhead

Kenneth Toliver

Sounds like a plan and I do appreciate you helping me figure this one out. Full disclosure; I previously only shot flats and flat darks, no bias. So this is the first time I've ever tried to integrate using bias (subtracted from the flats).

I will send you some screen shots tonight of statistics on each type of frame.

BlockHead

Ah... uncalibrated flats were NOT be helpful.

good luck.

-the Blockhead

Kenneth Toliver

I calibrated the flats using a master bias and a master dark (optimized/calibrated). 

I also attempted putting all the raw frames (lights, flats, darks, and bias) into WBPP and got the same output.

Kenneth Toliver

I'm closing this topic with information in the event others search for this type of problem but cannot find help.

The flats were always calibrated. My concern was whether I was calibrating them properly for blue-sky dawn flats. In the end, there really should be no difference aside from perhaps using percentile clipping.

I calibrated the flats 3 different ways including the time tested simple subtraction of bias. With 30 seconds on this camera, there is no reason to worry about dark current at that exposure length. Master darks and bias also look good. I always ended up with this gradient regardless of approach.

I am now convinced the gradients are due to high clouds that I can see when blinking through all the calibrated frames. Initially, I was only blinking them by night taken, but when I ran them in a long sequence, I could see lots of fluctuation in light levels.

BlockHead

Thanks for following up!

-the Blockhead

Kenneth Toliver

Hi Adam; 

I know you have little interest in this thread, but I have hit a wall. I've have been doing this 3 tears and have never come up against this "rainbow gradient". I have also always trusted PI and never really was much interested in the math.

I think I understand now what you meant when you said the image is "crushed". I ran statistics on my frames (I debayered all so that I could get similar stats to my integrated images) and would appreciate it if you could take a brief look and tell me if something jumps out at you.

Looking at RED alone, the single uncalibrated sub shows MIN: 494 and MAX: 65404 while my integrated image shows MIN: 84 and MAX: 64862. And the integration that I ran without any flats, only a master dark frame, shows MIN: 14 and MAX 61338. Is this what you mean by crushed? I can't imagine how a fully stacked image can have lower values than the individual subs, but I'm no mathematician. Could this be normal?

BlockHead

Unfortunately this is a case where I would need to see the data from the beginning.

If you want to continue to work on this... here is what you can do...I cannot take all of the it.

But let's go another round. If I am reading your history correctly you made some non-optimal choices for ImageIntegration. You did not do normalization, you turned the weighting off, and you used LargeScale Rejection. This is... a bad recipe. Am I reading this wrong? Your integration choices certainly could be spoiling the party...

Let me know..

-the Blockhead

(you can post a screenshot of your ImageIntegration)

Kenneth Toliver

Hi Adam. Thanks for the reply. You know, I've probably gone thru this stack 10 times since I posted that first XISF and, frankly, I was tweaking a lot without being very consistent. Let's re-baseline.

WBPP kept crashing with memory errors upon integration. I've taken some steps to fix that issue including buying a few more sticks and increasing my page file. I am going to reset WBPP back to 100% defaults and see if I can get through a clean run, with normalization on, then I will post back with a clean XISF.

BlockHead

Also, consider lessening the number of threads that PixInsight can use in parallel. 

You will likely want to limit this if you are experiencing crashes.

-the Blockhead

Kenneth Toliver

Problem resolved. I got WBPP to run the full stack, including LN, with no issues. Unfortunately, it's the same rainbow gradient. I'm uploading both a screenshot of the integration and the log. Any thoughts would be appreciated.

Additionally, here is a one-drive link in case you would like to look at the XISF

BlockHead

The noise doesn't look right to me.

It just looks wrong.

But without processing it myself from scratch I guess I cannot tell.

You would need to prove at each step everything makes sense. You are welcome to post each of the intermediate files (jpeg screenshots) with values shown. You would need to post your complete set of WBPP panels. You would also need to post your Normalizatoin and IMageIntegration settings. 

Alternatively. I can look a the data itself. But I just can't do that for free- not enough time. 

Another guess... there is a certain level of noise you might not be beating...the readnoise might be killing you here because the exposures are short.

Again, I don't know.

-the Blockhead

BlockHead

Just to be clear... looking at the final integrated image usually isn't that helpful other than to identify something isn't optimal.

-the Blockhead

Kenneth Toliver

I fully understand, and appreciate the feedback. I'm going to think about this a bit, but you are correct; the noise just doesn't look right.

I'm going to go with the fact that the transparency that night was just poor. Given that, I will assume that the next target, under clear skies, should be better. AND I intend to extend the exposure times.