Re: [ferret_users] EOF memory problem

[Ansley Manke <ansley.b.manke@xxxxxxxx>]

Hi Zhen,
Ferret allocates memory at a couple of different times.  The Ferret SET 
MEMORY/SIZE= command allocates a big block of memory which is then 
shared by most Ferret operations. In addition, though, the external 
functions framework, which is where the EOF functions were implemented, 
do separate memory allocations for work arrays that they use. Then that 
memory is released when the function finishes.

This means that if one allocates as much memory as possible with SET 
MEMORY/SIZE=, there is not enough memory available for the external 
function to use. Try a smaller SET MEMORY command for Ferret -- enough 
that it can load the 176 by  66 by  120 data field, plus some more for 
Ferret work space, but smaller than 500. The documentation for the EOF 
functions will be changed to clarify this.

I was able to start up the function with a SET MEM/SIZ=200  and a 
variable that is 144 by 73 by 120.  
A separate problem may arise with large problems like this one. There is 
currently an upper limit of 4000 on the number of eofs that the function 
would compute (the size of the spatial dimensions NX * NY), yielding 
errors like this

Bailing out of external function "eof_space":
       Number of EOFs too large:   10512 Reduce NX*NY. Max is 4000
**ERROR: : error in external function

We have a fix for this, and the limitation will be removed in the next 
Ferret release.

Ansley


zhen.li@gsfc.nasa.gov wrote:

> Dear Ferret Users:
>
> I've searched the Ferret Users' mailing archive regarding the EOF 
> memory problem before I sent this
> email. However, I still think that Ferret should be able to handle my 
> data set because I've regridded it to an array
> of [176  66   1 120]. But, I got an error message:
>
> ERROR in efcn_compute() allocating 540655544 bytes of memory
>       work array 5:  X=1:135163886, Y=1:1, Z=1:1, T=1:1
>  **ERROR: : error in external function
>
> I tried to allocate as much as memory as I can, but the maximum amount 
> I can allocate on my
> machine is 500.00 Mwords of memory. I wonder if there is any way to go 
> around this problem?
> I am using Ferret v5.81.
>
> Thanks,
>
> Zhen