[Thread Prev][Thread Next][Index]

Re: [ferret_users] calculation of monthly means from 5-day means



Hi Janeen -

Imagine that the data contains a strong periodic signal with a period of 31 days, and for simplicity that all months are 31 days long. Suppose the data are given on 15-day intervals, starting with day 1 of month 1. Then month 1 will have 3 values (days 1,16,31), but month 2 will have only 2 (days 15 and 30). If the signal is sinusoidal, with minimum on day 1 of month 1, then month 1 will sample two low values and one high, and month 2 will sample one high and one low. Thus the average of values within month 1 will be lower than that in month 2, even though they should both be average zero since we have a purely periodic signal. That would be aliasing (the poorly-sampled high frequency contaminates the low). But if the values are interpolated to daily, then smoothed with a filter that removes periods shorter than a month, then subsampled once/month (shouldn't matter which day the subsampling is done), then all months will have the same value, as they should.

The example shows that choosing a filter is crucial! In particular a boxcar filter (@sbx) will do a very bad job in this case! (It will produce an oscillation with the opposite phase, known as "ringing"; see attached plot and simple example!). However, a triangle filter (called Parzen (@spz) for silly historical reasons) will be much better (see plot). In general the spectral properties of the triangle filter are far superior to a boxcar in reduced ringing. In choosing a filter, it is always a good idea to make a few plots comparing the original and filtered time series to sheck.

Billy K

Example of ringing with a boxcar filter:

yes? define axis/x=1:16:1 xax
yes? let xx=x[gx=xax]
yes? let pi=4*atan(1)
yes? let signal=sin(pi*(xx-.5))
yes? plot/line/sym=27 signal,signal[i=@sbx:3],signal[i=@spz:3]

The reason the ringing occurs is because the boxcar average is unweighted: (i=1+ i=2 + i=3)/3, namely 2 highs and one low, the second is over 2 lows and one high, etc. The triangle filter weighting removes this (particular) problem. This is a pathological example, but many strange things can occur with filtering of inadequately-sampled data. it is always essential to plot and check!

GIF image

On Dec 22, 2006, at 8:51 AM, Janeen wrote:

Hi William,

If you wouldn't mind explaining a bit more, could you expand on what you mean by aliasing? I'm trying to follow along on this example.

Janeen


William S. Kessler wrote:
Hi Wim-Paul -

For 5-day data, taking the average of values that fall within a given month would introduce some aliasing.

Perhaps a better way to do this would be to interpolate the data to daily, then smooth with a monthly filter, then subsample at the 15th day of each month:

define axis/t=tstart:tend:1/t0=.../unit=day tday ! choose tstart,tend as the time limits of the data
let tempfl = temp[gt=tday] ! interpolate (linearly by default)
let tempflsm = tempfl[l=@spz:31] ! a filter with half-power point of about 37 days
define axis/t=15-monthstart-yrstart:15-monthend:yrend:1/t0=.../ unit=month tmon ! will get the warning below
! *** NOTE: /UNIT=MONTHS is ambiguous ... using 1/12 of 365.2425 days
let tempmon=tempflsm[gt=tmon] ! subsample to monthly

Billy K

On Dec 22, 2006, at 7:21 AM, Wim-Paul Breugem wrote:

Dear ferret users,

I have a dataset of 5-day means. The corresponding dates on
a standard calender are given for the third day of the 5-day interval.
As an example I show you here a subset of the data:

yes? list/i=1/j=1/k=1/l=1:20 temp

03-JAN-1958 12 / 1: 19.04
08-JAN-1958 12 / 2: 20.35
13-JAN-1958 12 / 3: 21.18
18-JAN-1958 12 / 4: 21.57
23-JAN-1958 12 / 5: 21.53
28-JAN-1958 12 / 6: 21.75
02-FEB-1958 12 / 7: 21.94
07-FEB-1958 12 / 8: 22.03
12-FEB-1958 12 / 9: 22.39
17-FEB-1958 12 / 10: 22.48
22-FEB-1958 12 / 11: 22.43
27-FEB-1958 12 / 12: 22.63
04-MAR-1958 12 / 13: 22.92
09-MAR-1958 12 / 14: 23.00
14-MAR-1958 12 / 15: 22.98
19-MAR-1958 12 / 16: 22.99
24-MAR-1958 12 / 17: 23.01
29-MAR-1958 12 / 18: 23.08
03-APR-1958 12 / 19: 23.10
08-APR-1958 12 / 20: 23.15

Does anybody know how to calculate from this the corresponding monthly
means? Thus, how do I obtain the monthly means for Jan 1958, Feb 1958, etc.?
Note that the 5-day means are usually not aligned with the edges of the
months, so the 5-day means should not all have the same weight in the
monthly averaging.

I'm looking forward to a solution for this problem!

With best regards,

Wim-Paul Breugem

--Wim-Paul Breugem KNMI, Oceanographic Research Dep.
P.O. Box 201, 3730 AE De Bilt
The Netherlands
tel./fax: +31-30-2206700/2202570
homepage: http://www.knmi.nl/~breugem
-------------------------------------------------------------------- ---------------------------------------------------------------
Zie ook/see also: http://www.knmi.nl/maildisclaimer.html

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
William S. Kessler
NOAA / Pacific Marine Environmental Laboratory
7600 Sand Point Way NE
Seattle WA 98115 USA

william.s.kessler@noaa.gov
Tel: 206-526-6221
Fax: 206-526-6744
Home page: http://www.pmel.noaa.gov/people/kessler/

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
William S. Kessler
NOAA / Pacific Marine Environmental Laboratory
7600 Sand Point Way NE
Seattle WA 98115 USA

william.s.kessler@noaa.gov
Tel: 206-526-6221
Fax: 206-526-6744
Home page: http://www.pmel.noaa.gov/people/kessler/



[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement