Black clipped pixels after stretching

shannon foye

Any idea what's causing a large amount of pixels to be blacked clipped after stretching despite the clipped pixels having values extremely close to the surrounding pixels?

I'm seeing this phenomenon with my narrowband and broadband master images. I see the problem both with and without a pedestal added to the subs. I tried adding an increasing amount of pedestal to subs, up to 1000, but the issue still persists. 

The master OIII image shown below can be downloaded at:

https://www.dropbox.com/scl/fi/mhzitu77guurzaqj6entq/OIII-165-NSG-pedestal-not-sub.xisf?rlkey=lr00hsr7m70uwwdm04ayajfpw&st=mw0fmk2d&dl=0

The integration time of the OIII image is 13.75 hours, consisting of 165 5 minute subs, taken with a QHY268 at 56 gain and 25 offset. I used NormalizeScaleGradiant and did not subtract the 1000 pedestal at ImageIntegration.

Attached below are screenshots this master OIII image before and after stretching with a simple STF / HT stretch. Some of the screenshots shows the pixel values of the neighboring pixels before and after stretching. In the last screenshot, the clipped pixels have been converted to red using the PixelMath function that Adam has used in his videos.

The pixel that gets black clipped after stretching had a value of 1118 out of 65536. The two pixels to the left and right of the clipped pixel had values of 1119 before stretching. The 1118 value pixel had a value of 0 after a STF stretch. The 1119 value pixels to the left and right of the clipped pixel had values of 1836 and 2905 after stretching.

Applying a small amount of NoiseXTerminator on the linear master image will reduce the amount of pixels that are black clipped but there are still a lot left to deal with. One solution is to apply CosmeticCorrection to the stretched master image. This may eliminate all black clipped on one master image, such as this OIII. However, CosmeticCorrection failed to eliminate many of the clipped pixels on my SII master image.

BlockHead

When you stretch your image (using HT)... is the black level set to zero?

It just seems like you are clipping  when you stretch the image. 

Your shadows show a non-zero value in HT. 0.01706864 .

Why did you choose this instead of zero? (I am assuming this is what you used to stretch the image.)

-the Blockhead

shannon foye

Adam,

Thanks for looking at this.

I did not deliberately choose a non-zero shadow value in HT. That is the result of dragging the STF settings of the linear image onto the bottom of HT and then applying HT to the image.

I believe that I've discovered why I am only seeing this problem occur recently. By comparing NoiseXTerminator results from 2022 and now, it appears that previous versions of NXT eliminated black-clipped pixels while the current version does not. And before NXT existed, any black-clipped pixels were removed by applying MLT after stretching to eliminate noise at the pixel level (i.e., remove the first layer of the multiscale transform), as you often recommended in your videos.

Regarding my suspicion that previous versions of NXT removed black-clipped pixels that the current version is leaving behind, I opened a 2022 PixInsight Projec to see if a previous NXT version achieved different results. When I clicked back in an image's history, I could see that the STF-to-HT stretch did black-clip pixels but those zero-value pixels were all eliminated during NXT (2022 version) noise reduction that was applied after stretching.

I made a clone of the stretched 2022 image and applied the current version of NXT to it. The current version of NXT did not remove the exact same black-clipped pixels that the 2022 NXT version did remove. However, when I clicked back and forth in the 2022 image history (before and after NXT), the same black-clipped pixels were being removed by NXT. See screenshots below.

I then wondered why I did not see this problem of black-clipped pixels standing out after noise reduction prior to NoiseXTerminator being released. I then remembered that (prior to NXT) one of the first steps I would take after stretching an image (as suggested by you) would be to remove the noise at the pixel level by stripping away the first layer of multiscale transform in MLT.

So, if the current version of NXT does not remove black-clipped pixels after stretching, my work around is to apply MLT (removing the first layer) before applying NXT. See screenshots below.

BlockHead

I think you mixing a few ideas.

1. Yes, the behavior of NXT may have changed over time.

2. The use of MLT is now pretty much an obviated method considering tools like NXT and others that are available now. I honestly don't know what to do with videos like this. The information is good and useful... but at the same time it is also now better to use processes like NXT.

3. Regardless of whether NXT takes care of the dark pixels...I think you should try something different. When you use autoStretch (before you drag it to HT)- this is NOT a divine god-given optimal result. It is a mathematical answer that is often good. Before you the STF over to HT... take the black point (left caret) and set it zero (or some small number that does not show you black pixels). Then apply NXT or whatever you want. 

I think your real issue is simply the initial stretch. Without having those dark pixels (that is displaying the background with that much contrast) using NXT on the stretched image will be easier.

I would also add... you can use NXT on the image *before* you stretch it as well. But take care not to clip anything before you make the stretch permanent.

-the Blockhead

shannon foye

Thanks Adam. I do rely on a straight STF autostretch too often. I am slowly going through your more recent videos to learn other methods of stretching, such as GHS.