Re: [ferret_users] Percentile along time axis for gridded data

[Russ Fiedler <russell.fiedler@xxxxxxxx>]

Hi Peng,

I did this in the dim distant past by saving the sorted data to a file and then picked the appropriate values out.

use coads_climatology
let numpoints=`sst,return=lend`
let numlats=`sst,return=jend]`
let/title="sorted indices" sorted_by_time=sortl(sst[l=1:`numpoints`])

! I found that I ran out of memory if I tried to sort too much for some GB sized datasets. It depends on the size of your dataset.
! You might be able to make larger windows or even do it in one go.

save/j=1/jlimits=1:`numlats`/clob/file=sorted.nc sorted_by_time
repeat/j=2:`numlats` save/app/file=sorted.nc sorted_by_time

can var sorted_by_time
use sorted

! Make a variable with time indices

let index_2d=0*x[g=sorted_by_time] + 0*y[g=sorted_by_time] + l[g=sorted_by_time]

! Create an integrating kernel for the percentile wanted 1 for the limit we want and missing elsewhere.

!25 percentile say. Make sure at least 1 valid point is used

let k25 = if l[g=index_2d] eq max(int(0.25*ignore0(sorted_by_time[l=1:`numpoints`@ngd])),1) then 1

! Show the indices used in the sorted data. Note that missing values are accounted for.


shade k25[l=@loc:1]

! Multiply by SST and sum  up

let sst_k25 = k25[gl=sst[d=coads_climatology]@asn]*sst[d=coads_climatology]

let/title="sst at 25 percentile" sst25=sst_k25[l=1:`numpoints`@sum]

shade sst25


Cheers,
Russ


On 30/03/16 05:32, Ge Peng - NOAA Affiliate wrote:

Found this message showing how to find percentiles for 2-dimensional gridded data:

http://ferret.pmel.noaa.gov/Ferret/Mail_Archives/fu_2004/msg00632.html

However, I would like to compute quantiles/percentiles along my time axis at each grid cell of the 2-dimensional gridded data. I.e., for each x and y, the percentiles are done along the time axis with valid data points. (We can ignore the z dimension for now.)

I could take the time series at each grid cell using nested repeat loop in x and y dimensions, following the above example to sort the data and place the ordered data onto the tiled axis.

It does not sound very efficient to me. Has anyone done something similar in ferret? Is there a better way to do this, perhaps without nested repeat loops in both x and y directions?

Appreciate any help.

--- Peng

--

Ge Peng, Ph.D
Research Scholar

Cooperative Institute for Climate and Satellites, NC (CICS-NC)

North Carolina State University (NCSU) and

NOAA’s National Centers for Environmental Information (NCEI)

Center for Weather and Climate (CWC)

151 Patton Ave, Asheville, NC 28801
ge.peng@xxxxxxxx
o: +1 828 257 3009
f:  +1 828 257 3002

Following CICS-NC on Facebook