[Thread Prev][Thread Next][Index]

Re: [ferret_users] EOF in ferret



Hi Nitin et al.,

There are simple methods to obtain just a few of the leading principal components, EOFs, singular values, or singular vectors. For example, search google for "partial svd" or "truncated svd" or "fast svd". One elegant algorithm (implemented in Python) is here:

   http://atpassos.posterous.com/fast-truncated-svd-using-random-projections

Billy is right to caution against the blind use of EOFs. They're great at extracting an efficient basis for variance; not so great at extracting physical modes.

Andrew

On 12/05/2011 05:04 AM, William S. Kessler wrote:
Unfortunately there is no way to find just the low-mode EOFs. A matrix must be inverted to find its eigenvalues, then those are sorted to find their size order. As far as I know, there is no way to find just a few of the eigenvalues of a matrix.

However, the routine used in Ferret is more efficient than many in that it recognizes that an N by M matrix has at most min(N,M) non-zero eigenvalues. It uses that fact to do the inversion in the most efficient order, not wasting CPU by finding extra zero eigenvalues.

Possibly a matrix-oriented tool (Matlab) would be faster in some situations, since there are ways to determine some characteristics of an input matrix before inverting it, then choosing among a menu of techniques for doing the inversion depending on those characteristics. The routine used by Ferret, which is based on the Numerical Recipes subroutine SVDCMP, does not do this.

I suggest you do some reading on EOFs, to find out what they can and cannot do, what they do well and poorly, before simply cranking your data through this calculation, especially since you have a large data set that will take a lot of time to calculate. EOFs are not a technique that should be applied blindly. A good reference for this is Bretherton, Smith and Wallace (1992, J.Clim., 5(6) 541-560). Also the discussion in Numerical Recipes SVDCMP section is very valuable for understanding the mathematics of inverting large matrices.

EOFs are not always the best way to extract signals from a data set. The most important thing to recognize is that EOF patterns are not physical! They may or may not have physical meaning. It is also possible to find a single EOF that mixes two signals, or conversely that one signal is spread among two or more EOFs. It is essential to understand what these techniques do to know if they are appropriate for the problem at hand.

One way to explore your data before doing a large calculation is to regrid to a coarser grid, then try the EOF on the smaller data set first.

Billy K



On 05 Dec 2011, at 4:09 AM, nitin patil wrote:

Dear ferret users,
I have calculated the eof as steps given on http://ferret.pmel.noaa.gov/static/Demos/ef_eof_demo/ef_eof_demo.html but it takes so much of time because it calculates lot of eof about 200 to 500 thats don't need.
Is their is any way to calculate only first 4 eof with varience and its pcs?

--
Regards,
Mr. Nitin Patil
National Climate Centre,
India Meteorological Department,
Pune - 411005, INDIA.




[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement