Re: [ferret_users] 21 yr running correlation

[Andrew Wittenberg - NOAA Federal <andrew.wittenberg@xxxxxxxx>]

Sandeep et al.,

Running correlations are easy to compute in Ferret.  If your signals are A and B, then:

! define the running time-averaging transformation,

! e.g. 21*12 grid points (21 years, if time step is 1 month)

def sym trans "t=@sbx:`21*12`"

let a2 = a^2; let b2 = b^2; let prod = a*b

let a_b_cov = prod[($trans)] - a[($trans)]*b[($trans)]

let a_b_corr = a_b_cov/((a2[($trans)] - a[($trans)]^2)*(b2[($trans)] - b[($trans)]^2))^.5

That's it -- no need for external files or long lists of terms.

Here's an expanded example, so you can understand what's going on:

! Define some example signals, A and B, whose correlation varies in time.

define axis/t=0:8000:1/unit=months/t0=1-jan-0001 time_ax

let pi = 3.14159

let tval = t[gt=time_ax]

let a = sin(2*pi*tval/100) * sin(2*pi*tval/4000)

let b = sin(2*pi*tval/98) * sin(2*pi*tval/2000)

! define the running time-averaging transformation

def sym trans "t=@sbx:`21*12`"

let a_ave = a[($trans)]

let b_ave = b[($trans)]

let a_squared = a^2

let b_squared = b^2

let a_std = (a_squared[($trans)] - a_ave^2)^.5

let b_std = (b_squared[($trans)] - b_ave^2)^.5

let a_b_prod = a*b

let a_b_cov = a_b_prod[($trans)] - a_ave*b_ave

let/title="running correlation" a_b_corr = a_b_cov/(a_std*b_std)

plot a, b, a_b_corr

Best regards,

Andrew

On Tue, Aug 28, 2018 at 3:01 PM, William S. Kessler <william.s.kessler@xxxxxxxx> wrote:

This could be done in a single Ferret session, most conveniently if the running length isn't too long. All the quantities needed for the calculation can be found from A_bar, B_bar, Asq_bar, Bsq_bar, AB_bar, where A and B are your variables and _bar is taken over the running interval.

These can be defined manually by terms such as (schematically)

let a_bar = ( a[l=@shf:(-n)] + a[l=@shf:(-n+1)] + ... + a[l=@shf:-1] + a + a[l=@shf:1] + ... + a[l=@shf:(n-1)] + a[l=@shf:n] )/(2n+1)

Thus, a_bar is a function of t (and possibly x,y,z).

Similarly for b, a^2, b^2, ab. They're just sums. If the line length is too long, the sum above can be broken into separate variables, then summed.

Then find your variances and correlations algebraically from standard expressions using these 5 sum terms.

It might be a pain to type out (a good text editor would help), but would let you do this in one script without intermediate writes.

Billy K

> On Aug 28, 2018, at 6:47 AM, Satyesh Ghetiya <satyeshghetiya@xxxxxxxxx> wrote:
>
> Hi Sandeep,
> 
> Do you have, for each year, correlation r in XY dimension ? I mean r for each grid point of XY for each year ? Also can you elaborate the dimensions of two data sets ?
>
> On Tue, Aug 28, 2018 at 11:48 AM, Sandeep Mohapatra <sandeepmohapatra21@xxxxxxxxx> wrote:
> Hi,
> can anyone tell me how to do 21 yr running correlation in ferret between two data sets.
>
>
>
>
>
> Regards,
> G Satyesh,
> India.
>
>
>
> On Tue, Aug 28, 2018 at 6:16 PM, Ryo Furue <furue@xxxxxxxxxx> wrote:
> On Tue, Aug 28, 2018 at 4:31 PM saurabh rathore <rohitsrb2020@xxxxxxxxx> wrote:
> Dear Ryo,
>
> Are you saying that the plot of such running correlation and trends will be like a time series ?
>
> I'm sorry, Saurabh, Sandeep, and other Ferret users.  I've begun to think that my idea was wrong.  I thought @SBX was the solution, but it doesn't keep the same mean value over the window, in formulae like
>
>       u' = u - mean(u).
>
> So, indeed the correct solution would be complicated.  I'd like to know if there is an elegant solution.
>
> Ryo
>
>
>
>
> --
> G Satyesh
>