[Thread Prev][Thread Next][Index]

[ferret_users] memory leak?




Dear Ferret:

It seems to me that there is a memory leak in ferret.

I have a list of 639 files, each has a 2D field and is about 4MB. I just want to join them all together on a new time axis, so I have a time series of 2D fields. But I run out of memory after 135 fields.

Things I've tried:

set memory/size=512

Even though I have 3GB of RAM, I can't set it to larger than that. Our system administrator contacted ferret about this but never got anywhere.

I've tried to patch the leaks:
Here is the script I tried:
!******************************************************************************
! Start of script:
!------------------------------------------------------------------------------
! names will hold the 639 file names to be read:
!------------------------------------------------------------------------------

let names = {spawn:"ls hh_msla_oer_merged_h_*"}

!------------------------------------------------------------------------------
! setup & clobber an empty NetCDF file
!------------------------------------------------------------------------------
SET DATA `names[i=1]`
DEFINE AXIS/T=15627:20093:7/UNITS=DAYS taxis

DEFINE GRID/LIKE=grid_0001/T=taxis grid_aviso

LET/TITLE="AVISO Sea Surface Height Anomaly"/UNITS=m \
ssha = (1/0) * x[G=grid_aviso] * y[G=grid_aviso] * t[G=grid_aviso]

SET MEMORY/SIZE=512

SAVE/CLOBBER/FILE=($out_file)/RIGID/LLIMITS=1:639 ssha
!------------------------------------------------------------------------------
! variable definitions
!------------------------------------------------------------------------------

LET dummie = x[G=grid_aviso] * y[G=grid_aviso] * t[G=grid_aviso]
! (change ssha from cm to m)
LET ssha_tmp = grid_0001/100
!------------------------------------------------------------------------------
! now append the rest
!------------------------------------------------------------------------------

REPEAT/k=1:639 (LET a = names[i=`k`]; \
SET DATA/regulart `a`; \
DEFINE AXIS/T=15627:20093:7/UNITS=DAYS taxis ;\
DEFINE GRID/LIKE=grid_0001/T=taxis grid_aviso ;\
LET/TITLE="AVISO Sea Surface Height Anomaly"/UNITS=m ssha=RESHAPE(ssha_tmp,dummie[L=`k`]);\
SAVE/APPEND/FILE=($out_file)/RIGID/LLIMITS=1:639 ssha[L=`k`]; \
CANCEL DATA /ALL ;\
cancel memory ;\
CANCEL AXIS/ALL)
!******************************************************************************
! End of script:
!******************************************************************************


This leads to an error about insufficient memory.

You might wonder why I keep redefining the axes. Well, if I don't I run into problems after 35 files! Below is the most basic script:

!------------------------------------------------------------------------------
! now append the rest
!------------------------------------------------------------------------------

DEFINE AXIS/T=15627:20093:7/UNITS=DAYS taxis
DEFINE GRID/LIKE=grid_0001/T=taxis grid_aviso

REPEAT/k=1:639 (LET a = names[i=`k`]; \
SET DATA/regulart `a`; \
LET/TITLE="AVISO Sea Surface Height Anomaly"/UNITS=m ssha =
RESHAPE(ssha_tmp,dummie[L=`k`]); \
SAVE/APPEND/FILE=($out_file)/RIGID/LLIMITS=1:639 ssha[L=`k`])

!******************************************************************************
!--------------------------------------------------------------------------------

This leads to an error:
!-> LIST/FORMAT=CDF/APPEND/FILE=ssha_aviso_NA.cdf/RIGID/LLIMITS=1:639
ssha[L=32]
LISTing to file ssha_aviso_NA.cdf
add_connect: too many connects, cannot add another
** netCDF error: Attribute not found
could not open CDF output file:
ssha_aviso_NA.cdf



This is very frustrating. Any help would be appreciated.
Thanks!
Rob



Rob Scott
Research Associate
Institute for Geophysics
University of Texas at Austin
4412 Spicewood Springs Rd.
Building 600
Austin, TX
78759

Office: +1-512-471-0375


[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement