[Thread Prev][Thread Next][Index]

Re: [ferret_users] [FERRET]memory issues in ferret



Hi Ansley:

What you stated here is correct. Ferret has run out of memory right at the regriding point. It can not handle the data of 700*390*43*120 word. So, I have to break down the data set into smaller segments. I'll read the documentation under "mode desperate" as you suggested.

Thank you for your help!

Zhen

At 04:23 PM 11/17/2005, Ansley Manke wrote:
Hi Zhen,
What do you mean when you say that the @VAR calculation "Does not work"? Is this the point in the script where you run out of memory?

Ferret will let you define all the variables you want, but it does not try to load data until you give it a command that asks for some action. Plotting commands, or a LIST or SAVE, or LOAD are the commands that trigger Ferret to go and try to get the data and do calculations. If the calculation involves several operations, like the regridding and then rho_un function and then zaxreplace Ferret makes a stack of operations and carries them out in sequence. I don't see any of the "action" commands in the piece of the script you've included.
You can try inserting LOAD commands into a script to see whether Ferret can execute the calculation at a particular point in the script.

define axis/x=120E:295E:1/units=degrees x1deg
define axis/Y=45S:20N:1/units=degrees y1deg

let salt_1deg=salt[GX=x1deg@ave, GY=y1deg@ave]
LOAD salt_1deg

I'm sure you'll run into the memory error right there. Ferret won't be able to load 700*390*43*120 words of data for any calculation. Have you looked at the documentation under "mode desperate"? I admit I haven't used that myself but it might be a way to get Ferret to break up the calculations into chunks.

Ansley

zhen.li@gsfc.nasa.gov wrote:

Dear Ferret Users:

Lately, I've often run into the memory limit in ferret. I am trying to process a multi-year model output (see the script
below). The script fails at T=@VAR transform for both original, fine grid (1/4 x 1/6) and regridded coarse (1x1) grid.

The problem occurs after I executes "std_var_245=salt245[T=@VAR]^0.5", because std_var_245 still have the same dimension as salt245 with l=120. T=@VAR transform doesn't work for both the fine and coarse grid.

On the other hand, T=@VAR transform does work if I just use it on "salt[l=1,T=@VAR] " without the calculation of sigma and saltonsigma. I believe that after calculation of rho_un and ZAXREPLACE, the memory has been used up. So, T=@VAR transform can not be carried out. I don't know how to fix this problem.

Any help is greatly appreciated.

Below is my script.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
use ts_9099 [monthly salinity and temperature data for pacific ocean from year 1990 to 1999]
[SALT Salinity 1:700 1:390 1:43 1:120
TEMP Potential temperature 1:700 1:390
1:43 1:120]
Note: the data set is too large, so I have to regrid to 1deg, but even with 1 deg data set,
I still run into the memory problem,

define axis/x=120E:295E:1/units=degrees x1deg
define axis/Y=45S:20N:1/units=degrees y1deg

let salt_1deg=salt[GX=x1deg@ave, GY=y1deg@ave] >>>dimension [176 66 43 120]
let temp_1deg=salt[GX=x1deg@ave, GY=y1deg@ave]

let sigma=rho_un(salt_1deg,temp_1deg,0)-1000;
define axis/z=23:26:0.5 axden;

let saltonsigma=ZAXREPLACE(salt_1deg, sigma, z[gz=axden]);

let salt245=saltonsigma[z=24.5];
let std_var_245=salt245[T=@VAR]^0.5

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>


--
This message has been scanned for viruses and dangerous
content by GMAO's MailScanner, and is believed to be clean.


--
This message has been scanned for viruses and dangerous
content by GMAO's MailScanner, and is believed to be clean.


[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement