What data should I keep for the future?

Eric Munters

Hello all,

I am trying to figure out what data I should save for the future. In the future, after photographing the same object again, I want to add it later to the already obtained data already processed in WPBB. But there is so much data in WPBB that I think is unnecessary for the future. What I do like to avoid is having to redo everything every time I have obtained new data. This prolongs the process enormously.  I shoot with a Mono camera. Perhaps Adam has already made a video about this? 

Haider

Hi Eric,

I tend to save all my data on an external hard drive. I often re visit these as i learn more and can combine my images for a better result-each photon counts and its no surprise when combine the data for a better image,

I hope this makes sense?

regards,

Haider

Ron Murray

I have a similar question, but ALL MY DATA  isn't putting the question to bed for me. :). By all my data, are we meaning just the captured lights and calibration frames? or do you mean the calibrated frames, the delayered frames, etc... as well?

I was thinking that I would just be saving final images as well as the raw data frames (lights, darks & Flats) so If I ever wanted to reprocess, add data, or try new techniques, I would be starting from the beginning and not the miiddle, but it would save a TON of space not keeping the intermediary files.  

Now im too inexperienced to say if this thinking is correct, but it intuitively seems like the right way to archive this data efficiently without losing anything but maybe a little time to restart from the beginning rather than save many many gigabytes of data unnecessarily.  

Anyone think my thinking is incorrect or missing something?  

Ron

BlockHead

Hi all,

Yeah, I am not a great one to ask. I do save everything (yes, even the intermediate files. 

This is particularly useful if I want to make a video, share, or find an error... or redo.

Yes, you would save space... but NOT time. It depends on what you are doing. By saving the intermediate (calibrated/registered) files... it can sometimes save a lot of time to add more, share or whatever.

I spent heartbeats generating those files. 

Anyway, my perspective. I seems you guys have the right idea...you just need to decide what is more important space... or time balanced by expense.

-the Blockhead

John Hughes

After running WBPP and having processed the image I would create the following folders as a minimum to retain data for future use;

- A folder named Data into which all my subs go. If I imaged over multiple nights then within the Data folder will be sub-folders labelled Session_date.

- A folder named Master Calibration Files. Into this goes the master flats for each channel and master darks for each exposure length generated by WBPP. Again this may contain sub-folders if multiple flats were used for different sessions.

- A folder named Master Lights into which the final light frames (the files you would process in PixInsight to generate your image) outputted by WBPP for each channel or OSC are kept.This allows you to have a go at re-processing the image or use new tools/techniques in the future.

- A folder named Final Image into which a JPEG and XISF for the fully processed image is kept.

I would then delete the WBPP folder and anything else and save the above to an external drive. Hope that helps.