Re: [ferret_users] Sample variance and computing running variance

[Russell Fiedler <russell.fiedler@xxxxxxxx>]

Hi,

For a running variance. say we have a simple 1d time series var and we want a window +/-5 wide

let var=sin(l[l=1:100]/10)
let win=5
let var2d = xsequence(var)+0*var         !  Reshape var to x axis and make 2d
let maskedvar=if (  l ge i - win and l le i + win ) then var2d  ! window data
let runvar=maskedvar[i=1:100@var]            ! take variance in i pseudo direction
let runmax=maskedvar[i=1:100@max]        ! test with max
let runmaxferret=var[l=@smx:11]        ! new ferret smoother for comparison should be same as runmax
plot/l=1:100 var,runvar,runmax,runmaxferret

Russ


On Wednesday 28 September 2011 06:47, Ansley Manke wrote:
> Hi -
> 1)You make a good point about the variance calculations.  We'll look into this.
> 
> 2)For a running variance script, I have one simple idea.  This message,
> http://www.pmel.noaa.gov/maillists/tmap/ferret_users/fu_2010/msg00685.html
> 
> talks about computing a running maximum over a time window (in more recent times we have the SMX and SMN transformations for smoothing using min or max. The above message came before those existed).   It just writes each result to a file and then reads that back in.  It's maybe not elegant, but is straitforward.
> 
> -Ansley
> 
> 
> On 9/27/2011 9:16 AM, jagadish karmacharya wrote:
> Dear ferret users,
> 
> I have 2  questions regarding computing variance.
> 
> 1. sample and population based variance ( equivalently standard deviation):
> 
> I have noticed that stat command gives standard deviation based on a sample where as var command gives variance based on a population. My assumption is its customary to calculate sample based variance unless otherwise stated but why does ferret compute variance based on population with var command? I think it is possible to get sample based variance e.g. for my 1-D data with l=140
> list/precision=6 v1[l=@var]*v1[l=@ngd]/(v1[l=@ngd]-1)      !  = 7005.65 (this is still slightly different from sample based variance computed from other method e.g. excel gave 7002.42 )
> 
> Isn't there a way to compute sample based variance in more straight forward manner and why its not the default option (with var)?
> 
> 2. Computing running variance:
> 
> I am using boxcar smoother transformation (e.g. v1[l=1:140@sbx:31]) to compute running mean but I can't think of a way to compute running variance (standard deviation) in similar manner. My guess is this would involve looping over time but I am not able to work out a way.
> 
> Please advice. Thanks in advance.
> 
> Best regards,
> Jagadish
>