[Thread Prev][Thread Next][Index]

Re: Best strategy for long-time series


You have touched on a problem that we in the development group haven't yet developed much expertise in.  I would like to rephrase the question to the group (and John Caron):

Can the DODS aggregation server efficiently serve up time series that require opening hundreds or thousands of files?

Here are various potential solutions to the problem, each with its own drawbacks:

-- Jon

The advantage of this

Jean-Francois PIOLLE wrote:

I would like to serve with LAS a long time series of SST maps over
atlantic : we have up to 8 maps/a day (every 3 hours but some are
sometimes missing). This time series is daily updated with new maps. So
far, we have more than 4500 netCDF map files to serve.
I tried to serve them through a DODS aggregated server and set up the
LAS (v6.0) to access the data through this DODS server. But it appears
then that the first access to a map (actually the first access to a DODS
dataset) is really slow, exceeding the LAS default time-out. This is
because (I guess) each time the DODS catalog is updated, a DODS dataset
has to be re-aggregated and this first step is very time-consumming.
My question is : is DODS the best strategy to serve a long time-series
through LAS?  It seems that a Ferret descriptor file can't be used here
since it is limited to 500 files. What other strategy? I guess many
people have already experienced this problem...


Jean-francois Piolle

Jean-Francois PIOLLE
CERSAT  French ERS Processing and Archiving Facility
29280 Plouzane

Tel.:  (+33) 2 98 22 46 91   email: jfpiolle@ifremer.fr
Fax:   (+33) 2 98 22 45 33   WWW:   http://www.ifremer.fr/cersat


[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP
Contact Us | Privacy Policy | Disclaimer | Accessibility Statement