**To**:**ansley.b.manke@xxxxxxxx****Subject**:**Re: [ferret_users] R-square problem****From**:**Ryo Furue <furue@xxxxxxxxxx>**- Date: Tue, 20 Oct 2009 13:02:39 -1000 (HST)
- Cc: a.rodriguez@xxxxxxxxx, ferret_users@xxxxxxxx
- In-reply-to: <4ADE1F01.1090706@xxxxxxxx>
- References: <C70343C8.15D8%a.rodriguez@xxxxxxxxx> <4ADE1F01.1090706@xxxxxxxx>
- Sender: owner-ferret_users@xxxxxxxx

Hi Ansley and Ana, | Ana sent me her data, with P and Q. These are variables that look | quite well correlated but with one being of larger magnitude than the | other. This is causing a loss of numeric accuracy; Ferret only | operates in single-precision, and so the variations in the data of | smaller magnitude are overwhelmed by the larger numbers. I guess precision is being lost when large number is subtracted from another large number: let pp = p*p let pave = p[t=@ave] let pvar = pp - pave*pave !! Large - Large If the average of p is smaller than its standard deviation, the result will be better. So, set var/name=p_in p ! I am going to re-define p set var/name=q_in q ! I am going to re-define q let p = p_in - 100000 ! min(p) = 100000 let q = q_in - 1000 ! min(q) = 1000 or even let p = p_in - p_in[t=@ave] let q = q_in - q_in[t=@ave] I think this change will give a better result. We have shifted the origins of the p and q axes so that the averages of the data fall on the origins. For the correlation and the slope of regression, the origins of the axes don't matter. The intercept would have to be shifted back to the original coordinates. Regards, Ryo

**References**:**Re: [ferret_users] R-square problem***From:*Ansley Manke

- Previous by thread:
**Re: [ferret_users] R-square problem** - Next by thread:
**[ferret_users] Subsample in time**

Contact Us

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Privacy Policy | Disclaimer | Accessibility Statement