[ferret_users] Tips and Tricks to avoid memory problems in Ferret

[Jaison Kurian <jaison@xxxxxxxxxxxxxxxxxx>]

Hi Ferreters,
               We all must have seen the following error message from 
Ferret, quite a number of times....

       **ERROR: insufficient memory: ???????? words were requested.

Here is few tips/tricks to avoid this error. With this tricks, you will
also be able to :
              - say goodbye to "set memory/size=???mb" command
              - save some of your precious time
              - monitor the progress of your script .... interesting??

Tips
====
 1. Memory error happens when the entire XYZT grid is written out in a single
         stretch, or when some regridding/transforming operations are 
         performed on multiple axes.

 2. Therefore, a straight solution is to split the writing out process into
         multiple slices, along a PROPER axis.

 3. There is no neat method (i guess so) to do such a slicing if you intend
         to display the result of a complex calculation/regridding. A work
         around is to write out the result to an intermediate file, using 2. 
         above, and read it back to create a display! 

 4. Remember that Ferret is not an "immediate evaluator" by default, and
         it does so only when you force with grave accents (``). Hence you 
         can write all the calculation steps before the writing out part.

Tricks
======
  From Ferret/NetCDF point of view, all the axes will not behave in the same
      manner. Hence different cases are discussed separately.
 
  case 1 : calculation on X/Y/Z axes, and NOT on T axis
  -----------------------------------------------------
  Say the calculation involves only the X/Y/Z axes, so that it is safe to work 
  on time "slices". The TIME axis have a special nature that, always it will be
  the RECORD dimension. Hence you can append data without any special features
  requested for the output file. The best solution will be doing the writing
  out part within a REPEAT loop on "L".


       sp rm -f outfile.nc                 ! keep it here
       let l_start = `myvar,r=lstart`
       let l_end   = `myvar,r=lend`
       let slices  = l_end - l_start + 1       
       REPEAT/L=`l_start`:`l_end`:1 (;\
           save/file=outfile.nc/APPEND/quiet  myvar ;\
           say " Time slice  `(L-l_start+1)`/`slices`" ;\
       )


  case 2 : calculation on T axis
  ------------------------------
  If the calculation is done on T-axis (time), then we have to slice the 
  writing process along any other axis, which does not affect the calculation.
  The priority goes like Z, Y and X. The procedure remains same for all three,
  since none of them are RECORD axis. For the same reason, if we have to open
  a new file for writing out, we have to specify the corresponding abstract
  LIMITS in the first step and respective index of data in subsequent steps. 
  If we are slicing along Z-axis, the Ferret commands will be

 
       sp rm -f outfile.nc              ! keep it here
       let k_start = `myvar,r=kstart`
       let k_end   = `myvar,r=kend`
       let slices  = k_end - k_start + 1       
       save/file=outfile.nc/KLIMITS=1:`slices`/K=1/CLOBBER/quiet myvar 
       say " Depth slice  1/`slices`" 
       REPEAT/K=`k_start+1`:`k_end`:1 (;\
           save/file=outfile.nc/APPEND/K=`K`/quiet  myvar ;\
           say " Depth slice  `(K-k_start+1)`/`slices`" ;\
       )


  Please note that the "K=1" step is done separately, since it needs the 
  "/KLIMITS" qualifier.


Cheers !!

Jaison

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.