[Thread Prev][Thread Next][Index]

Re: [ferret_users] Making n/n+1 correction from var_n.jnl script



Hello again,

I explored around a bit and noticed few things and though it would be relevant to share in discussion.

1) variance.jnl script compute population variance and covariance so n/n+1 correction is required (i.e. need to run var_n script) for computing sampling variance and covariance (generally, we are interested in sample statistics). However, correlation is same for population or sample statics, so if one is interested in computing correlation there is no need to run var_n script (and the value doesn't alter even if the var_n script is run)

2) I think the var_n script give correct result only when the data/ variable is in correct format. I think the problem is rooted in definiting L1 = L. So it would be better to revise this script and update the FAQ.

Let me state the isses with an example (I have shown result only for covar but same should hold for variance)

case 1: var_n doesn't give correct result for absctract axis
 let X1 = TSEQUENCE({1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20})
 let Y1 = TSEQUENCE({10,12,40,30,25,56,75,65,66,67,101,80,85,88,180,100,88,120,206,105})
 let p=x1;let q=y1
 go variance
 list covar
          231.6
 go var_n
 list covar
          231.6            ! doesn't chage.

! This is because NDNM1 is not correct as its taking value from abstract axis
list nm1, ndnm1
          1.000E+07    1.000

! Case 2: However, if the variables are assigned to a time axis we get correct result

def axis/t=1:20:1 tax
let p1=x1[gt=tax];let q1=y1[gt=tax]
let p=p1;let q=q1
list covar
          243.8            ! Correct result (though nm1and NDNM1 still have same value (which I guess is due to difference rounding off issue in display and computation))

! This is clearer if the variables are saved to a new file
save/file=p1q1.nc p1,q1

! In new ferret session
use p1q1.nc
let p=p1;let q=q1
go variance
list covar
          231.6
go var_n
list covar
          243.8            ! Correct result
! Now values of NDNM1 is correct   
 list nm1, ndnm1
          19.00    1.053

! case 3: However,one is interested in sample variance/ covariance for subset of data the result is again wrong
! New ferret session (or can continue fromabove)

use p1q1.nc
 let p=p1[l=1:10];let q=q1[l=1:10]
 go variance
 list covar
          59.10
 go var_n
 list covar
          62.21          ! Wrong result, correct (excel)  value is 65.67
! This is because value of nm1 is still 19 (instead of correct value of 9)
 list nm1, ndnm1
          19.00    1.053

With the problem identification, I think updating var_n script should not be that difficult ....

Thanks,

Jagadish

From: jagadish karmacharya <j_karmacharya@xxxxxxxxx>
To: "ferret_users@xxxxxxxx" <ferret_users@xxxxxxxx>
Sent: Tuesday, July 16, 2013 8:11 PM
Subject: [ferret_users] Making n/n+1 correction from var_n.jnl script

Hi,

I wanted to ask this question since long, but have somehow missed till now.

In the FAQ for correlations and variances (http://ferret.pmel.noaa.gov/Ferret/faq/correlations-and-variances) it is advised to run var_n.jnl to make the n/n+1 correction. I wonder if the correction is intended for variances and covariance only or also for correlation? The FAQ suggest noting the change in correl but end up depicting same value (as that before running var_n). Is this refering to some other change? - I assume not. Running var_n after variance script doesn't change value of correlation (but it does for variances and covariance).

Also the correction factor NDNM1 consider entire series in its computation, so if someone is interested in correlation for subset of time series between two variables, I think NDNM1 would not represent correct correction factor.

Please advise.

Many thanks in advance

Jagadish



[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement