[Thread Prev][Thread Next][Index]

Re: [ferret_users] Memory question



That grid is huge (about 8.6x10^9 gridpoints). You can do it with the method below but this is going to make about a 35Gb file. It's hard to imagine why you would want that, and it will be cumbersome to use. Much better to make a series of single-timestep files, and a descriptor file (see documentation/mail archives for descriptor files).

Netcdf files can be appended easily along the time axis (for other axes, see the documentation on netcdf).

First write a single time-point to describe the (x,y,z) grid, then append the others, in chunks as large as can go in memory:

save/l=1/file=filename.nc yourvariable
save/l=2:100/file=filename.nc/append yourvariable
save/l=101:200/file=filename.nc/append yourvariable

.... etc ....

I'd also be a little careful about specifying your REGION with only 2 decimal places. Are you sure that your specification will get all the 5-min gridboxes exactly? Don't want to leave out the ones near the Dateline. And since this appears to be the whole world, why do you need to make that REGION specification at all? (In a case like this, it would be better to do it with indices I,J if it really needs to be done).

BK

On 19 Mar 10, at 9:11 AM, Jonathan Winter wrote:

Hi all,

I am trying to regrid a 0.5 deg dataset to 5-min and then write it out as a NetCDF file. I use the following commands:

use 030410_init_wsoi.nc
use vegtype_5min.nc
show grid vegtype[d=2]
show grid wsoi_lay[d=1]
define grid/X=GJR1/Y=GJR1/Z=GOL1/L=GOL1 veg_grid
let wsoi =  wsoi_lay[d=1,G=veg_grid]
set variable/title="Soil Moisture" wsoi
set region/X=-179.96:179.96/Y=-89.958:89.958N/Z=0.1:4/T=6941:7275
save/file=wsoi.nc wsoi

At that point I get:
 **ERROR: insufficient memory: 18662400 words were requested.
 *** NOTE: You can use SET MEMORY/SIZE=xxx to increase memory.
*** NOTE: The "Memory use" section of the FERRET Users Guide has further tips.

So I:
set memory/size=20000000

And I get:
*** NOTE: internal overflow expressing =20000000 Mwords as words. Restoring previous memory size.
 Cached data cleared from memory

If I try to set memory/size=1000000000 FERRET says it has done it, but then I run out of memory again if I try to write out:
 **ERROR: insufficient memory: 671846400 words were requested.
 *** NOTE: You can use SET MEMORY/SIZE=xxx to increase memory.
*** NOTE: The "Memory use" section of the FERRET Users Guide has further tips

Any thoughts on what is going on or how I may be able to do this? Thank you in advance for the assistance.

Best,

Jonathan



[Thread Prev][Thread Next][Index]

Contact Us
Dept of Commerce / NOAA / OAR / PMEL / TMAP

Privacy Policy | Disclaimer | Accessibility Statement