[ferret_users] A note about Ferret's IF statement

[Ansley Manke <ansley.b.manke@xxxxxxxx>]

Hi all,
Just a note about a conversation we've been having in our hallways.  Even experienced Ferret users get caught unawares now and then. 

The IF statement in Ferret has a couple of uses. One is for setting up conditional control in a script

   IF condition THEN ...

The second is to define a mask

   LET mask = IF elev GT 0 THEN 1    ! If elevation > 0 then 1 else missing
   LET masked_temp = temp* mask      ! Temperature where elevation positive

For all IF statements, the "TRUE" condition includes valid nonzero data values, and the "FALSE" condition includes the value zero and also invalid data values.  This all works as I think we'd expect for conditional control in scripts,  but unexpected results can arise in using IF for masking.

In effect there are two different flavors of "existence" of a variable in Ferret.

One is reported by STAT, and in the "number of good" (@NGD) and "number of bad" (@NBD) syntax. This is the usual "existence" of geophysical  variables, which can be zero and still be valid. Only explicitly MISSING values "do not exist" by this test.

The other flavor found in IF statements for defining masks (IF A THEN B). In this case, a zero value of variable A is taken as "not existing", and the result of the command is that variable B will be labeled missing at those points in addition to any points where A matches its missing-value flag. The results can lead to calculations that are not what the user intended.

There is a simple workaround (see definition of D in the example  below), but some users will not know that this is essential, and even users who do know it forget, because when one routinely  works with zero values as existing, it is non-intuitive. We're posting this to the list to point out that these different behaviors occur. This is noted in the documentation but perhaps it could be made more prominent. 

If you are masking with IF/THEN, and you intend to test for valid-data but not zero-values, then you must explicitly state that you are testing against missing values, or else zero values will be taken as missing.

----------------------------------------------------
A simple example:

yes? let a = {0,1,2,,3}    ! one zero and one explicitly bad value
yes? let b = {5,6,7,8,9}
yes? let c = if a then b

yes? list a,b,c
             X: 0.5 to 5.5
 Column  1: A is {0,1,2,,3}
 Column  2: B is {5,6,7,8,9}
 Column  3: C is IF A THEN B
             A     B      C
1   / 1:  0.000  5.000   ....
2   / 2:  1.000  6.000  6.000
3   / 3:  2.000  7.000  7.000
4   / 4:   ....  8.000   ....
5   / 5:  3.000  9.000  9.000

==>> While the user might think that zero is a perfectly good value of  A, the IF/THEN mask finds it invalid, and C is blanked where A is  missing, AND where A=0.

==>> On the other hand, "STAT a" finds only the explicitly missing  value to be blank, and the @NGD test finds the same:

yes? stat a
 
             {0,1,2,,3}
             X: 0.5 to 5.5
             Y:  N/A
             Z:  N/A
             T:  N/A
             DATA SET: N/A
 
 Total # of data points: 5 (5*1*1*1)
 # flagged as bad  data: 1
 Minimum value: 0
 Maximum value: 3
 Mean    value: 1.5 (unweighted average)
 Standard deviation: 1.291

yes? list a[i=@ngd]
             VARIABLE : {0,1,2,,3} (# of points)
             X        : 0.5 to 5.5 (number of valid)
          4.000

==>> There is a simple workaround :

yes? let d = if a ne `a,return=bad` then b

yes? list a,b,d
             X: 0.5 to 5.5
 Column  1: A is {0,1,2,,3}
 Column  2: B is {5,6,7,8,9}
 Column  3: D is IF A NE -1.000000E+34 THEN B
             A     B      D
1   / 1:  0.000  5.000  5.000
2   / 2:  1.000  6.000  6.000
3   / 3:  2.000  7.000  7.000
4   / 4:   ....  8.000   ....
5   / 5:  3.000  9.000  9.000