Re: [ferret_users] Making n/n+1 correction from var

Hello again,

I explored around a bit and noticed few things and though it would be relevant to share in discussion.

1) variance.jnl script compute population variance and covariance so n/n+1 correction is required (i.e. need to run var_n script) for computing sampling variance and covariance (generally, we are interested in sample statistics). However, correlation is same for population or sample statics, so if one is interested in computing correlation there is no need to run var_n script (and the value doesn't alter even if the var_n script is run)

2) I think the var_n script give correct result only when the data/ variable is in correct format. I think the problem is rooted in definiting L1 = L. So it would be better to revise this script and update the FAQ.

Let me state the isses with an example (I have shown result only for covar but same should hold for variance)

case 1: var_n doesn't give correct result for absctract axis

let X1 = TSEQUENCE({1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20})

let Y1 = TSEQUENCE({10,12,40,30,25,56,75,65,66,67,101,80,85,88,180,100,88,120,206,105})

let p=x1;let q=y1

go variance

list covar

231.6

go var_n

list covar

231.6 ! doesn't chage.

! This is because NDNM1 is not correct as its taking value from abstract axis

list nm1, ndnm1

1.000E+07 1.000

! Case 2: However, if the variables are assigned to a time axis we get correct result

def axis/t=1:20:1 tax

let p1=x1[gt=tax];let q1=y1[gt=tax]

let p=p1;let q=q1

list covar
243.8 ! Correct result (though nm1and NDNM1 still have same value (which I guess is due to difference rounding off issue in display and computation))

! This is clearer if the variables are saved to a new file

save/file=p1q1.nc p1,q1

! In new ferret session

use p1q1.nc

let p=p1;let q=q1

go variance

list covar

231.6

go var_n

list covar

243.8 ! Correct result

! Now values of NDNM1 is correct

list nm1, ndnm1

19.00 1.053

! case 3: However,one is interested in sample variance/ covariance for subset of data the result is again wrong

! New ferret session (or can continue fromabove)

use p1q1.nc

let p=p1[l=1:10];let q=q1[l=1:10]

go variance

list covar

59.10

go var_n

list covar

62.21 ! Wrong result, correct (excel) value is 65.67

! This is because value of nm1 is still 19 (instead of correct value of 9)

list nm1, ndnm1

19.00 1.053

With the problem identification, I think updating var_n script should not be that difficult ....

Thanks,

Jagadish

From: jagadish karmacharya <j_karmacharya@xxxxxxxxx>
To: "ferret_users@xxxxxxxx" <ferret_users@xxxxxxxx>
Sent: Tuesday, July 16, 2013 8:11 PM
Subject: [ferret_users] Making n/n+1 correction from var_n.jnl script

Hi,

I wanted to ask this question since long, but have somehow missed till now.

In the FAQ for correlations and variances (http://ferret.pmel.noaa.gov/Ferret/faq/correlations-and-variances) it is advised to run var_n.jnl to make the n/n+1 correction. I wonder if the correction is intended for variances and covariance only or also for correlation? The FAQ suggest noting the change in correl but end up depicting same value (as that before running var_n). Is this refering to some other change? - I assume not. Running var_n after variance script doesn't change value of correlation (but it does for variances and covariance).

Also the correction factor NDNM1 consider entire series in its computation, so if someone is interested in correlation for subset of time series between two variables, I think NDNM1 would not represent correct correction factor.

Please advise.

Many thanks in advance

Jagadish

Re: [ferret_users] Making n/n+1 correction from var_n.jnl script