Re: eof calculus and missing values

[Mick Spillane <spillane@xxxxxxxxxxxxx>]

> Is it possible to know how Ferret manages the presence of missing values in
> the calculation of the EOF.

Hi Antonio,
	You may get a more detailed explanation of this from one of the
others in the group or by looking at the code for these external
functions.  However here is a brief statement of how it is done.

	Basically what needs to be done is to get the eigensolution to
a covariance matrix. Such a matrix can be calculated even when some
data values are missing, provided that sufficient cross products are
available to make all elements in the matrix meaningful.  The latter
is the point of the PERCENT parameter in eof_stat, eof_space, and
eof_tfunc - if a grid point has so many missing values that its cross
products are unreliable it is dropped entirely from the analysis.
	So now there is a covariance matrix, slightly imperfect, but
for which a set of eigenvectors can be computed, using a standard
method (such as "jacobi.f" in Numerical Recipes, Press et al.).  The
trick comes in computing the "time functions".  Some of the values
of the individual data series, needed for a normal evaluation are
missing.  The ferret eof routines use an objective method, based on
some unpublished notes of Dudley Chelton and similar to the algebra
in Davis (1976) and Chelton et al. (1982) to find the best, in a least
squares sense, value of the time function at times where there are
missing values.  Not provided in the Ferret version, though available
via the underlying algorithm, is a measure of the "expected" error at
those points in time where the objective estimation is necessary.

Hope this helps.  The references below should give you the flavor
of the objective method employed.

Mick

Davis, R.E., 1976 Journal of Physical Oceanography, 6: 249-266.

Chelton, D.B., P.A. Bernal, and J.A. McGowan, 1982, Journal of
	Marine Research, 40(4): 1095-1125.