Re: Vectorize a XYT box or, convert a 3D var to 1D

[Mick Spillane <Mick.Spillane@xxxxxxxx>]

Hi Paulo,
	Depending on the accuracy you need in the median and quartile
values, a simple "hunt and peck" method within ferret may be a viable
method. It avoids the need to sort which probably is the most time
consuming task (see below for suggestions based on the NON-SORTING MDIAN2
routine in Press et al "Numerical Recipes" ).

        For a "hunt and peck"example to locate the median and quartiles of
the levitus climatology try this demo:

use levitus_climatology
let ngood=temp[x=@ngd,y=@ngd,z=@ngd] ! shows there are 718725 valid data
let mask=if(temp gt guess)then 1
let frac=mask[x=@ngd,y=@ngd,z=@ngd]/ngood

! now the task is to find a value for "guess" to make frac=0.5 (for the
! median or 0.75,0.25 for the quartiles

let guess=5 ; list frac   ! this gives 0.4803, not a bad start and it took
                          ! about a dozen guesses (each taking ferret 5s
                          ! or so to get "close enough"
let guess=4.53 ; list frac ! 0.5003
!-----------------------------------------------------------

It's not an elegant solution but avoids write/sort/read and may be of use
for the occasional user.

If you are going to be a frequent median calculator of VERY LARGE arrays
of values, then it might be worth while looking into iterative, NON-SORT
methods such as MDIAN2 in Press et al. "Numerical Recipes".  Press says
that this routine converges in O(logN) passes.  I could see implementing
the method inside ferret to improve the guessing strategy of my "hunt and
peck" idea above, with ferret saving the need to write an external file
and deal with missing values in your data.  And doing it in ferret you
could proceed to the quartiles (medians of the lower and upper halves of
the distribution) by masking the variable based on the median
	let median = ... ! the value located above
	let loval=if(temp lt median)then temp
	let mask=if(loval gt guess)then 1
	let ngood=718725/2 ! half the original ngood
	... proceed to find guess to make "frac" = 0.5 again
	    and ditto for
	let hival=if(temp gt median)then temp

Good luck,
Mick

|!!! Mick.Spillane@noaa.gov !!!|
|__Room 2070 Bldg#3 NOAA/PMEL__|
|____Phone_:_(206)526-6780_____|