Re: [ferret_users] Missing values - interpolation and smoothing

[Hein Zelle <hein.zelle@xxxxxxxxxxxxx>]

Dear Szymon Roziewski,

> I have such a problem, namely I have data hs as an array (132,145) from
> hs.cdf file with missing value set as -999.
> Then I read those data and try to interpolate (regridding with smoothing).

Ferret seems to be doing exactly what you ask, and it is working
properly.  I think you misunderstand how the smoothing affects your
data: see below.   Your code can also be made somewhat simpler.

> use hs.cdf

If you then try a 

yes?  shade hs

you'll see that the HS variable is already properly defined, and the
missing data is correctly set: you only see data over water, nothing
is plotted over land.  You don't need to do anything further with the
missing data flag.

> DEFINE AXIS/X=-5.666666667:5.25:0.010416667 xfine
> DEFINE AXIS/Y=-2.0:10.0:0.010416667 yfine
> DEFINE GRID/x=xfine/y=yfine fine_grid

That part is fine.

> let/bad=-999 var_hs=hs[i=1:132,j=1:145,k=0,l=1]
> let/bad=-999 fine_hs=var_hs[gxy=fine_grid]
> LET/bad=-999 ubox_hs=fine_hs[x=@SBX:19,y=@SBX:19]
> LET/bad=-999 utaper_hs=ubox_hs[x=@SHN:13,y=@SHN:13]

Leave out all the /bad statements above: they're not needed.  You
don't need to specify the indices to var_hs if you want "everything".
In your case you can leave out var_hs completely.

> let fine_hs=hs[gxy=fine_grid]
> LET ubox_hs=fine_hs[x=@SBX:19,y=@SBX:19]
> LET utaper_hs=ubox_hs[x=@SHN:13,y=@SHN:13]

Try making plots of each individual variable:

> shade var_hs
> shade fine_hs
> shade ubox_hs
> shade utaper_hs

you'll see that fine_hs looks pretty good: ferret has interpolated
your coarse data to a finer grid.  it's already quite smooth there.

Note that the fine data does NOT cover exactly the same data as the
original: at the edges you loose some data.  This is because Ferret
refuses to extrapolate: beyond the center of the last coarse grid
point, you will not see any data.

The same thing happens with ubox_hs: because @sbx needs 19 data points
to average, it will drop the outer 9 points of your sea data: at those
points there are no valid 19 surrounding data points.

The same thing happens again with utaper_hs: the @shn smoother will
discard another 6 points of data on the edges of the sea.
Alltogether, you loose a LARGE edge of data along the boundaries this
way.

Solutions:

1)  only use 1 smoother at all times, and try to keep the amount of
grid points small.  Do you need smoothing at all?  fine_hs looks quite
neat to me.

2)  use a gaussian weight function to go directly from your coarse
grid (hs) to the fine grid with smoothing/averaging.

The function is called scat2gridgauss_xy, documentation is here:
http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/XPRESSIONS#_VPID_159

Look for section Chapter 3, 2.3.37 SCAT2GRIDGAUSS_XY

In your case, do something like this:

> let xpts = xsequence(0*hs+x[g=hs])
> let ypts = xsequence(0*hs+y[g=hs])     
> let datapts = xsequence(hs)
>
> let xscale=0.1
> let yscale=0.1
> let cutoff=3
>
> let hs_smooth = scat2gridgauss_xy(xpts,ypts,datapts,x[g=fine_grid],y[g=fine_grid],xscale,yscale,cutoff,0)
>
> shade hs_smooth

You'll see that hs_smooth "invents" data around the edges: the
gaussian bell extends beyond your original data.  You can remove that
extra data with a "sea mask", I'll use your fine_hs here:

> let hs_smooth_masked = if (fine_hs) then hs_smooth
> shade hs_smooth_masked

Play with xscale, yscale and cutoff to achieve more or less smoothing.
Try xscale=0.5 and yscale=0.5 to see the difference.  If plotting
takes very long, try reducing the density of your fine grid a bit.


Kind regards,
     Hein Zelle


-- 

Dr. Hein Zelle
Advisor Meteorology & Oceanography

Tel:    +31 (0)527-242299
Fax:    +31 (0)527-242016
Email:  hein.zelle@xxxxxxxxxxxxx
Web:    www.bmtargoss.com

BMT ARGOSS
P.O. Box 61, 8325 ZH Vollenhove
Voorsterweg 28, 8316 PT Marknesse
The Netherlands

----Confidentiality Notice & Disclaimer---- 

The contents of this e-mail and any attachments are intended for the
use of the mail addressee(s) shown. If you are not that person, you
are not allowed to read it, to take any action based upon it or to
copy it, forward, distribute or disclose the contents of it and you
should please delete it from your system. BMT ARGOSS does not accept
liability for any errors or omissions in the context of this e-mail or
its attachments which arise as a result of internet transmission, nor
accept liability for statements which are those of the author and
clearly not made on behalf of BMT ARGOSS.