Re: Slow Calculation Speed - Can it be Faster?

[hankin@xxxxxxxxxxxxxxxxxxx (Steve Hankin)]

	*************************************************************
	Ferret tip:  efficient reuse of previously calculated results
	*************************************************************

Hi Mark,

Ferret can efficiently reuse results that were previously cached in memory only
if the commands and definitions you give unambiguously specify the region of
interest.

In your example it means that you replace an action command like


	yes? go xcorrel tqs1 tqs2
with
	yes? go xcorrel tqs1[l=1:100,X=30W,Y=30N] tqs2[l=1:100,X=30W,Y=30N]


or alternatively replace a definition like


	yes? let tqs1=qc[d=1,l=@ave]
with
	yes? let tqs1=qc[d=1,l=1:100@ave,X=30W,Y=30N]


I hope this helps - steve

================================================

On Apr 11,  2:22pm, Mark Verschell wrote:
> Subject: Slow Calculation Speed - Can it be Faster?
> I have written a routine to calculate the correlation coefficient between two
> data series (code attached below). I can understand why some of the first
> calculations are slow, but once the values are calculated once, it appear
that
> everytime they are referenced after that they are calculated all over again.
Is
> there a way to prevent this?
>
> My calling routine is:
>    use "TS01/TS01dt"
>    use "TS20/TS20dt"
>    let qc1=qcdt[d=1]
>    let tqa1=qc1-qc1[i=@ave]   ! remove mean
>    let tqs1=tqa1[l=@ave]      ! average longitudinal profile
>    let tqn1=tqs1/0.00021232   ! normalize by standard deviation
>    let qc2=qcdt[d=2]
>    let tqa2=qc2-qc2[i=@ave]
>    let tqs2=tqa2[l=@ave]
>    let tqn2=tqs2/0.00011629
>    go xcorrel tqn1 tqn2
>
> XCORREL.JNL is:
>    let/quiet Xi = $1
>    let/quiet Yi = $2
>    let/quiet XiYi = Xi*Yi
>    let/quiet Xi2 = Xi*Xi
>    let/quiet Yi2 = Yi*Yi
>    let/quiet sXiYi = XiYi[i=@sum]
>    let/quiet sXi2 = Xi2[i=@sum]
>    let/quiet sYi2 = Yi2[i=@sum]
>    let/quiet mX = Xi[i=@ave]
>    let/quiet mY = Yi[i=@ave]
>    let/quiet nX = Xi[i=@NGD]
>    let/quiet eN = $3%nX%
>    list sXiYi, sXi2, SYi2, mX, mY, nX, eN
>    let/quiet sX2 = sXi2-nX*mX*mX
>    let/quiet sY2 = sYi2-nX*mY*mY
>    list sX2, sY2
>    let/quiet rXY = (sXiYi-nX*mX*mY)/(sX2*sY2)^0.5
>    list rXY                                        ! *** 1 ***
>    let/quiet w = 0.5*LN((1+rXY)/(1-rXY))
>    list w                                          ! *** 2 ***
>    let/quiet signif = w*(eN-3)^0.5
>    list signif
>
> First, I realize that in this particular case that since I have removed the
> mean and normalized by the standard deviation so that my correlation function
> could
> be greatly simplified, but I wanted the routine to be more general.  In the
> future I will likely add some flag that will indicate that I am using
>  normalized anomaly series and thus run simplified code. Right now I am
trying
> to understand a more general principle. In particular, in XCORREL.JNL at the
> line indicated by the comment *** 1 *** all the values used to calculate rXY
> have already been calculated once before (in the list statements), but it
still
> takes a long time to calculate. Furthermore, at the line in XCORREL.JNL
> indicated by the comment *** 2 ***, Rxy has already been calculated - but w
> takes forever to calculate.
>
> I tried inserting load/perm statements before the list statements to
hopefully
> get around this. Nope, the list statements took just as long so obviously the
> load/perm statements didn't do what I hoped.
>
> Lastly, I tried simplifying my calling routine to:
>    use "TS01/TS01dt"
>    use "TS20/TS20dt"
>    let tqs1=qc[d=1,l=@ave]  ! average longitudinal profile
>    let tqs2=qc[d=2,l=@ave]  ! average longitudinal profile
>    go xcorrel tqs1 tqs2
>
> This worked very quickly, so it must have something to do with the mutliple
> definitions in my calling routine. But I am not sure how to work around this.
> Any help would be appreciated!
>
> 	Thanks, Mark
>
> --
> -------------------------------------------------------------------------------
>
> 	      Mark Verschell (verschell@coaps.fsu.edu)
> 	      Center for Ocean and Atmospheric Prediction Studies
> 	      Florida State University
>               2035 E. Paul Dirac Drive
>               R.M. Johnson Bldg. - Suite 200
> 	      Tallahassee, FL  32306-3041
> 	      (904) 644-6532              (904) 644-4841 (fax)
>-- End of excerpt from Mark Verschell