GradientCorrection won't run

Comments

• BlockHead

  March 8

  This appears to be specific to your machine. It is basically saying there was a  memory issue.

  
  

  Two things to do..

  
  

  1. Reinstall PI. Just in case there is something corrupt about your installation.

  2. You need to check your memory if the above does not help. This appears to be a hardware issue.

  3. You can post to the official forum... but this isn't likely to bear useful fruit unless others have the issue.

  
  

  But hopefully #1 helps.

  
  

  -the Blockhead
• Edward Smith

  March 8

  Thank you.  Some additional information.  I didn't post the entire run because it does start to run the routine (it does the preprocessing).  When I turned on the Simplified model, it will run just fine so long as the Model Degree is set to 1.  However, if I try to run it on itself (in other words, I run it and then run it again on the corrected image), I get the error again.  I can play with all of the adjustments and so long as I run with the Simplified Model with Model Degree 1 on an un-GradientCorrected image, it runs fine.  This is, at least so far, the only module Process or Script I'm having issues with and none of the my other programs have run into a problem.
  
  A little more research indicates that the  the code C0000005 indicates that more than one piece of software is attempting to access the same memory address.  Some additional research and troubleshooting is going to be necessary I think.  In the meantime, I'll just use DBE which is running just fine.
• Edward Smith

  March 8

  Just for anybody else who is having this issue and getting the same code, additional research seems to indicate that this is some kind of a software issue that is usually corrected by either a Windows or other software program update.  Since I'm positive that PixInsight is fully updated, I'm searching through other software that may be the issue.  Since PixInsight uses the GPU, it is possible the issue could even be with the video drivers.  When I figure it out, I'll post here.
  
  In the meantime, I found a workaround that has GradientCorrection working after a fashion.  It isn't running normally, but I can use it more or less as Adam described.
• Wade Hilmo

  March 8

  I hate to disagree with Adam, but there is far too little information here to say with any kind of certainty that this is a memory issue.

  
  

  The exception that you are seeing, C0000005, indicates that the problem tried to read a value from a specific memory location, but there was no memory allocated at that address.  Think of it as the computer equivalent of trying to visit someone's house, only to find a vacant lot at the address you are using.

  
  

  In all honesty, software bugs are far and away the most common cause of this exception.  The problem is, that it's pretty much a general error.  Using the vacant lot analogy, you know that the lot is currently vacant.  You don't know if you were given the wrong address; or if the house used to be there, but was torn down; or whether the house just hasn't been built yet.  In the software world, there are nearly infinite ways to reach this error.

  
  

  This error, and the call stack (all the red stuff in your post under "Backtrace Information") are meaningful to the developer of the software, but in order to act on it, they typically need to be able to reproduce it on a machine that they can live debug.  The problem there, is that when you post to their forum, they are almost certainly going to say that they cannot reproduce it themselves, and can therefore not act on it.  Their logic is that if there is a bug in PixInsight, lots of folks will report it, and they'll also be able to reproduce it in testing.  If they are not seeing an obvious pattern, then they will conclude that the problem is specific to your machine.

  
  

  I did do a search for this exception in PixInsight with RtlCaptureContext2 on the stack, and found a good number of posts already on the PixInsight forum.  Predictably, the most common response from the PixInsight folks is that they need to reproduce the problem locally to act on it.  But I did read some things that gave me an idea.  And I should note here that not all of the reports are the same.  A proper explanation as to how I know that these similar call stacks are all different problems goes beyond what I could write concisely.

  
  

  It's possible that the system is failing a memory allocation due to memory exhaustion.  If that is the case, there is something that you can do about it.  Modern systems use something called a paging file as a sort of scratch paper when they don't have enough physical memory to handle an allocation.  The way that it works, is that when a program asks for memory when there's none left, instead of failing the request, they'll write some memory that hasn't been touched in a while to the paging file.  They can then free up that memory to handle the new request.  If the paging file isn't set up, or isn't large enough, it can lead to this exception.

  
  

  Based on your call stack, it looks like you are running on Windows.  I have included steps at the end of my response to check the page file size on Windows, and increase it if necessary.

  
  

  All that said, it is possible that Adam is right that you actually have a hardware problem. although it may not be memory.  I've been in this situation myself, after building a new computer a little over a year ago.

  
  

  When I first built the machine, I ran a bunch of stress tests on it, and they all passed.  I also installed the software that I planned to use, including PixInsight.  Everything on the machine ran great, except for PixInsight, which crashed so frequently, that I could not get a project to the point of having an integrated image.

  
  

  There were 3 different problems, all of which needed to be addressed before the machine ran PixInsight in a stable and reliable way.

  
  

  The first problem was that the operating system did not create a page file.  Any time a memory allocation failed, PixInsight would crash.  Creating a proper size page file made a world of difference, but didn't completely resolve the problems.

  
  

  The second problem was more difficult to run down.  It turned out that the manufacturer of my motherboard chose some very unfortunate default settings.  It was pushing too much voltage to the CPU in some situations.  I suspect that they did this so that they could advertise higher performance.  Fixing these settings corrected most of the remaining crashes, but I was still seeing them occasionally.

  
  

  I discovered the third problem quite by accident.  I sat down at the machine one day and found that it was turned off (I leave it running 24x7).  When I started it up, it booted but crashed almost immediately, even before trying to run PixInsight.  I was able to determine that the CPU was over heating and could not run for more than a couple of minutes at best.  I fixed the problem by completely replacing the cooling system.  After that, I stopped seeing crashes pretty much entirely.  I suspect that the cooling system was never operating properly and failed completely after a couple of months in use.

  
  

  And note nearly all of these crashes indicated an invalid memory access exception, just like you are seeing.

  
  

  I'm sorry that I can't give you a direct and actionable answer, but I wanted to give you the most accurate information about the problem that you are seeing.

  
  

  And finally, here are steps to check and adjust the page file on Windows:

  • Right click on the Windows start button and pick "System" from the menu.
  • On the System page, look at the links in the upper right and pick "Advanced system settings".
  • When the "System Properties" window comes up, pick the "Settings" button in the top box, labeled Performance.  If you don't see the performance settings, make sure that you are on the "Advanced" tab.
  • In the "Performance Options" window that comes up, select the "Advanced" tab.

  
  

  When you are on the "Advanced" tab in the "Performance Options" window, look at the bottom section, entitled "Virtual memory".  In particular, look at the "Total paging file size for all drives" value.  It should be at least as large as the amount of physical memory you have.

  
  

  If you need to increase the size of the page file, click the "Change" button.  In the next dialog, you might need to uncheck "Automatically manage paging file size for all drives".  When you do that, you can set a custom size.  As above, it should be at least as large as your physical memory.  And if you have a choice of drives, you want it on your fastest drive.  Note that you will need to restart the computer for this to take effect.

  
  

  PS:

  
  

  I also wanted to add a correction that no native PixInsight processes use the GPU, and that includes GradientCorrection.  The only modules in PixInsight that use the GPU are 3rd party ones.  Off the top of my head, that would include BlurXTerminator, NoiseXTerminator, StarXTerminator and Starnet++.  I seem to remember that this is at least one more, but it slips my mind at the moment.  In any case, the GPU is not involved in the case described here.

  
  

  Also, can you tell us about your workaround?  It might give me some other ideas about the root cause of your specific issue.
• Edward Smith

  March 10

  Wade/Adam,
  
  Thank you both.  I've done a couple of days of troubleshooting and scratching my head over this and finally figured out what was causing the problem even if I have no idea why.
  
  I looked at all of the software and everything was updated.  I read Wade's post but went ahead and ensured that all of the drivers were updated anyway.  I still had the issue.  Thinking back to my programming days, there are 3 things that can generally cause errors; Hardware Issues, Software Issues, and Data Issues.  Based on how PixInsight really stresses the computer in other operations leading up to the Gradient correction, I was having a hard time believing Hardware Issue, but, well, maybe.  I'd checked all of the Software I could think of and that wasn't the problem.  Maybe a Data Issue?
  
  So I went back and looked at the Image.  Remember that I got different errors if tried to run Gradient Correction after already running it (and Adam had mentioned that you can get strange results) and found that I had already run NoiseExterminator on the image.  So I went back and re-integrated the data and ran GradientCorrection on the Just Integrated (ie, no other process run yet) image and it ran perfectly.  I hit undo, ran NoiseExterminator, and then GradientCorrection and got the error.  I undid the NoiseExterminator and ran GradientCorrection again and it ran perfectly.  I haven't tried the same set of processes on a different data set, but I'm confident that I've hit on the problem.
  
  So, for whatever reason, it appears that GradientCorrection doesn't like to be run after NoiseExterminator.  The problem wasn't hardware or software related, it was data related.  
  
  Thank you both for your help.  I appreciate your time and effort.  Your information definitely helped my troubleshooting.  I'm happy to supply the files in question to anybody who would like to look at them.
• BlockHead

  March 10

  If repeatable... I am certain PI developers would be interested in seeing this.

  That is quite a find. 

  I can imagine some ways NXT might mess up MCG... but not Gradient Correction. 

  Perhaps in the analysis of the small scale structures... it finds none and tries to do something odd.

  No idea...

  
  

  -the Blockhead
• Wade Hilmo

  March 10

  Indeed.

  
  

  If you have data that can reproduce this problem on more than one machine, the PixInsight folks would probably love to get it.

  
  

  -Wade
• Edward Smith

  March 10

  Wade - I only tried my one machine, but it is repeatable on it.  I've only tried the one image as well, so I'm unsure if the problem is somehow related to that single dataset or not.
  
  Adam - I've been scratching my head over that as well.  To be honest, I know that data CAN cause issues like this, but I really didn't expect that this would fix the problem.  All I can think of is that the NXT somehow did something to the data that caused a null value at a pixel or some kind of discontinuity between pixels.  The camera this data was captured with has 2 hot columns and 2 dead columns that were corrected curing Cosmetic Correction in WBPP so I'm not sure if that plays into this or not  The message says that it runs the pre-processing just fine but then bombs during actual execution.  I'm actually impressed that PixInsight captures the failure without crashing the entire program.
  
  

  I do plan on trying this again with the next dataset I process to see if it is repeatable across images or if it was specific to this one.  I'll be posting on the PixInsight forumn and offering them the images.
  
  Mark
• Wade Hilmo

  March 11

  If I may offer a piece of advice for reporting the problem to the PixInsight folks...

  
  

  My suggestion would be to save the image that crashes PI after running NXT, but before running GradientCorrection as an XISF file and provide that in your report.  I would specifically not talk about the previous processing steps, especially NXT.  If they believe that NXT might be the problem, they will probably not investigate.

  
  

  If, however, you provide them with just the XISF file, and say something like "PixInsight crashes running GC on this image", they will likely be inclined to fix it.  The simpler your report, the more likely it will get traction.

  
  

  -Wade