Re: Large files access time

[J-M Epitalon <Jean-Marie.Epitalon@xxxxxxxxxx>]

Jon and Steve,

sorry. I mislead you. 
My file is 1.5 Gb (not Mb) !
and it is accessed through NFS.
I suppose this is why it is really slow.

Anyway, when accessed localy, it still takes half a minute to open it.
(3 to 4 mn thru NFS)

Since I want to keep the system architecture with NFS, I solved my
problem by splitting the file in several parts and using a MC file.

Please, what would you suggest as an alternative solution ?

Jean-Marie Epitalon
CERFACS
Toulouse, FR

On Fri, 2005-04-22 at 18:27, Jonathan Callahan wrote:
> Jean-Marie,
> 
> This seems inordinately slow for Ferret.  Especially for a file that is 
> only 1.5 Mb!  When this happens it usually means that the time axis is 
> marked as 'irregular' which means that Ferret has to read in the entire 
> time axis before it can begin.  We have found many cases where the axis 
> was actually 'regular' in spite of what the NetCDF attribute says.  If 
> this is the case then you can use ncatted to change the time axis 
> attribute and Ferret should open this dataset much more quickly.
> 
> 
> -- Jon
> 
> 
> J-M Epitalon wrote:
> 
> >hello,
> >
> >I have daily simulation output covering the period 01-01-1950 to
> >31-12-2005. It is in a NetCDF file that is 1.5 Mb large.
> >
> >When I access it through Ferret, it takes 2 to 4 mn to open it (execute
> >instruction "use").
> >
> >I read in NCO documentation that "Some random access operations on large
> >files on certain architecture are slow when using the NetCDF interface".
> >
> >That seems to be the problem. I could check that it is slow also with
> >other tools than ferret (Python, or NCO tools).
> >
> >Anyway, what solution would you suggest to work around this ?
> >Is using MC descriptors a good idea?
> >
> >Thanks
> >
> >Jean-Marie Epitalon
> >CERFACS
> >Toulouse, France
> >
> >  
> >
> 
>