Hello again,
I explored around a bit and noticed few things and though it would be relevant to share in discussion.
1) variance.jnl script compute population variance
and covariance so n/n+1 correction is required (i.e. need to run var_n script) for computing sampling variance and covariance (generally, we are interested in sample statistics). However, correlation is same for population or sample statics, so if one is interested in computing correlation there is no need to run var_n script (and the value doesn't alter even if the var_n script is run)
2) I think the var_n script give correct result only when the data/ variable is in correct format. I think the problem is rooted in definiting L1 = L. So it would be better to revise this script and update the FAQ.
Let me state the isses with an example (I have shown result only for covar but same should hold for variance)
case 1: var_n doesn't give correct result for absctract axis
let X1 =
TSEQUENCE({1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20})
let Y1 =
TSEQUENCE({10,12,40,30,25,56,75,65,66,67,101,80,85,88,180,100,88,120,206,105})
let p=x1;let q=y1
go variance
list covar
231.6
go var_n
list covar
231.6 ! doesn't chage.
! This is because NDNM1 is not correct as its taking value from abstract axis
list nm1, ndnm1
1.000E+07 1.000
! Case 2: However, if the variables are assigned to a time axis we get correct result
def axis/t=1:20:1 tax
let p1=x1[gt=tax];let q1=y1[gt=tax]
let p=p1;let q=q1
list covar
243.8 ! Correct result (though nm1and NDNM1 still have same value (which I guess is due to difference rounding off issue in display and computation))
! This is clearer if the variables are saved to a new file
! In new ferret session
use p1q1.nc
let p=p1;let q=q1
go variance
list covar
231.6
go var_n
list covar
243.8 ! Correct result
! Now values of NDNM1 is correct
list nm1, ndnm1
19.00 1.053
! case 3: However,one is interested in sample variance/ covariance for subset of data the result is again wrong
! New ferret session (or can continue fromabove)
use p1q1.nc
let p=p1[l=1:10];let
q=q1[l=1:10]
go variance
list covar
59.10
go var_n
list covar
62.21 ! Wrong result, correct (excel) value is 65.67
! This is because value of nm1 is still 19 (instead of correct value of 9)
list nm1, ndnm1
19.00 1.053
With the problem identification, I think updating var_n script should not be that difficult ....
Thanks,
Jagadish
From: jagadish karmacharya <j_karmacharya@xxxxxxxxx>
To: "ferret_users@xxxxxxxx" <ferret_users@xxxxxxxx>
Sent: Tuesday, July 16, 2013 8:11 PM
Subject: [ferret_users] Making n/n+1 correction from var_n.jnl script
Hi,
I wanted to ask this question since long, but have somehow missed till now.
In the FAQ for correlations and variances (http://ferret.pmel.noaa.gov/Ferret/faq/correlations-and-variances) it is advised to run var_n.jnl
to make the n/n+1 correction. I wonder if the correction is intended for variances and covariance only or also for correlation? The FAQ suggest noting the change in correl but end up depicting same value (as that before running var_n). Is this refering to some other change? - I assume not. Running var_n after variance script doesn't change value of correlation (but it does for variances and covariance).
Also the correction factor NDNM1 consider entire series in its computation, so if someone is interested in correlation for subset of time series between two variables, I think NDNM1 would not represent correct correction factor.
Please advise.
Many thanks in advance
Jagadish