Limits of WBPP Intelligence?

Nimbus7

I've watched the WBPP tutorials for managing keywords and such over multiple nights but my current SCHEME on the object I am presently imaging is somewhat different. I have already used WBPP to combine SESSIONS 1 and 2 into one master.  After examinging it I decided I wish to add an additional 6-8 hours of integration time.  If I add another two sessions of Lights for instance into WBPP as it stands (no clearing or resets),  will all the previous work in the Output Directory be ignored or overwritten?

My guess is that the previously "calibrated" files would be utlizied and that the weights will be invalidated since new files have been added.  Cache will not be purged unless recommended to do so.  Hoping to salvage as much previous processing time as possible.

I have also noticed in this version of WBPP that instead of choosing the desktop instance of CC but using the built-in option of CC that a separate directory is not created for such a choice.  The log gives the following error: [2024-09-29 11:08:45] * Performing cosmetic correction.
[2024-09-29 11:08:45] ** Warning: negative or insignificant pixel sample values detected after calibration.

No files are flagged with -CC as well.

Doesn't happen when creating the desktop instance of CC.

Thanks.

BlockHead

It will be neither ignored or overwritten. Actually WBPP never overwrites files.  If you run the new set of files... It will only calibrate the new files and not the old ones. You want this. You would change the registration reference to be the file that was chosen during the first run. Do not use automatic here. Now all of the already registered files will be skipped and only the new files should be aligned. I do not believe WBPP will reuse the same LN reference- because it will reanalyze including the new files. This particular step takes time and I do not think there is an easy way around this. Finally all of the images will be integrated. New LN references will be created as well as new LN (XNML) files. So you will see an increase of files in this respect (with the duplicates having a 1 appended).

Regarding the new CC- I explained this in my video on the subject. You may have missed my video. 

The new CC is a part of ImageCalibration. This is why you will not have _CC files.

---the Blockhead

Nimbus7

Thanks for confirming those processes Adam.  Back to the _CC question, this new CC version generates an error (see above) and in my case from the log the process was ignored.  I may go back and use your recommended _CC instance as before.

Thank you,

Nimbus

BlockHead

Ooooh!

That is telling. This would explain the LN issue as well. I assumed you looked at your data. 

It appears your calibrated data is bad. Which means something about calibration is suspect.

Troubleshooting 101...

-the Blockhead

Nimbus7

This is really getting nutz for me.  Anyway I cleared the WBPP output directory and cache.  Chose 5 lights to experiment on.  Ran a sequence and got the "** Warning: negative or insignificant pixel sample values detected after calibration".  Cleared the directories and ran a sequence without FLATs, still got the warning.  Cleared again and ran a sequence uncalibrated and no longer had a warning.  Well WBPP made the master flats and darks, don't know of anything I can change in that regard.

Before doing the above troubleshooting I went into a completed set of 60 images with the same kind of error in the log and the Master Light looked fine other than needing more lights to improve noise.

In addition I loaded in the resultant Process Container into the Script Editor and ran and viewed each process and none of those processes indicates the error which only shows up in the final log.

I guess now the question becomes what process in PI generates this error into the log?

I recently completed this image of M15 with the same Warning and got this result:

Sorry for this diffiulty.

BlockHead

Again you need to look the calibrated data. This is being measured- and it means there is oversubtraction which means the dark frames you are using are inappropriate, bad, or misconfigured.

Your masterLight is probably NOT fine. That is the point. 

If you are doing narrowband, then the pedestal is not correct. 

All of this indicates oversubtraction- and you need to find out the cause. 

Measure your raw light frame background...and then measure the average dark level. If the dark is a larger number... it cannot be correct.

-the Blockhead

Nimbus7

Thanks Adam.

These are OSC with no bias.

MasterDark= .008

Raw Light= .010

Calibrated Light= .006- .008

I dont see how Dark and Flat frames can be misconfigured if WBPP made them?  How could a user make WBPP do them right at default settings?

I ran the first five lights through Image Calibration manually with the same Master Dark and Flat and no warnings are generated in the Process Console.  Only when running WBPP does this occur in the beginning at the Calibration stage.  I don't know what to think at this stage so for now I will manually calibrate all 104 files and then hand them over to WBPP and see what happens.

Nimbus7

Just manually calibrated 104 Lights using the same Master Dark and Flat and added them into WBPP to complete.  No errors posted in Process Console.

BlockHead

You looked at ALL of the calibrated frames? 

The answer is there...

 It is saying there is oversubtraction. If LN later tries to use these frames as a reference... there will be issues.

There are no defaults in WBPP. 

You can post your panels using the diagnostic button and attaching them here.

-the Blockhead

Nimbus7

I Blinked all the calibrated frames enlarged, other than a few with different intensities and occasional roque hot pixel there was nothing that stood out wrong to my 76 year old eyes.

By defaults I mean a WBPP completely Reset fresh other than setting PSF SNR.

Sorry I am not aware of a diagnost button to post 104 panels.

Anyway as I mentioned earlier I ran Image Calibration externally from the script and all was well with no warnings in the Process Console.  The Master Light looked fine other than I will add another two hours for better SN..When Image Calibration ran within WBPP the Process Console immediately began flagging **Warnings.  I then loaded all the externally calibrated Lights into WBPP to complete as from the beginning and all was well.

----

In addition I contacted another User who just recently installed PI and completed his first set of images and had him check his WBPP Log.  Not only did his run have the same ** Warnings as mine but also included MetaData errors and some others.

Thanks again,

Bill