Hello again,
I explored around a
bit and noticed few things and though it would be relevant
to share in discussion.
1) variance.jnl script
compute population variance and covariance so n/n+1
correction is required (i.e. need to run var_n script) for
computing sampling variance and covariance (generally, we
are interested in sample statistics). However, correlation
is same for population or sample statics, so if one is
interested in computing correlation there is no need to run
var_n script (and the value doesn't alter even if the var_n
script is run)
2) I think the var_n
script give correct result only when the data/ variable is
in correct format. I think the problem is rooted in
definiting L1 = L. So it would be better to revise this
script and update the FAQ.
Let me state the isses
with an example (I have shown result only for covar but same
should hold for variance)
case 1: var_n doesn't
give correct result for absctract axis
let
X1 =
TSEQUENCE({1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20})
let
Y1 =
TSEQUENCE({10,12,40,30,25,56,75,65,66,67,101,80,85,88,180,100,88,120,206,105})
let
p=x1;let q=y1
go
variance
list
covar
231.6
go
var_n
list
covar
231.6 ! doesn't chage.
! This is because NDNM1 is not correct as its
taking value from abstract axis
list nm1, ndnm1
1.000E+07 1.000
! Case 2: However, if the variables are
assigned to a time axis we get correct result
def axis/t=1:20:1 tax
let p1=x1[gt=tax];let q1=y1[gt=tax]
let p=p1;let q=q1
list covar
243.8 !
Correct result (though nm1and NDNM1 still
have same value (which I guess is due to difference
rounding off issue in display and computation))
! This is clearer if the variables are
saved to a new file
! In new
ferret session
use
p1q1.nc
let p=p1;let q=q1
go variance
list covar
231.6
go var_n
list covar
243.8
! Correct result
! Now values of NDNM1 is correct
list
nm1, ndnm1
19.00 1.053
! case
3: However,one is interested in sample variance/ covariance
for subset of data the result is again wrong
! New
ferret session (or can continue fromabove)
use
p1q1.nc
let
p=p1[l=1:10];let
q=q1[l=1:10]
go
variance
list
covar
59.10
go
var_n
list
covar
62.21 ! Wrong result,
correct (excel) value
is 65.67
! This
is because value of nm1 is still 19 (instead of correct
value of 9)
list
nm1, ndnm1
19.00 1.053
With the
problem identification, I think updating var_n script should
not be that difficult ....
Thanks,
Jagadish
Hi,
I wanted to ask this question since long, but have
somehow missed till now.
In the FAQ for correlations and variances
(
http://ferret.pmel.noaa.gov/Ferret/faq/correlations-and-variances)
it is advised to run var_n.jnl to make the n/n+1
correction. I wonder if the correction is intended
for variances and covariance only or also for
correlation? The FAQ suggest noting the change in
correl but end up depicting same value (as that
before running var_n). Is this refering to some
other change? - I assume not. Running var_n after
variance script doesn't change value of correlation
(but it does for variances and covariance).
Also the correction factor NDNM1 consider entire
series in its computation, so if someone is
interested in correlation for subset of time series
between two variables, I think NDNM1 would not
represent correct correction factor.
Please advise.
Many thanks in advance
Jagadish