Hi Karl,
You’re right the crash is due to the output, thank you!
However… I can’t get ferret to read the data right.
When I try to write to netcdf is crashes with a signal 11 – I guess again an output issue…
But:
I got a little fortran program to read the data (gfortran compiled)
I give this below and what I think the ferret set data should look like.
The stats command works, but the numbers are not right.
Now I am really confused. Maybe this is an issue with our installation.
Our machine was recently upgraded. Let me check on this before anything else.
Maybe you could just give me your opinion whether I got the right ferret “set data…” which does the same as the fortran code.
There should not be any count numbers for the records as the file size fits exactly with the data.
Thank you very much!
Kind regards,
Joerg
This bit of fortran reads the data fine (the numbers are between 0 and 300):
program test
real*4,dimension(227964) :: x
integer i
open(10,file="GB_1km_197101.bin",ACCESS="DIRECT",RECL=911856,CONVERT="BIG_ENDIAN" ) ! 4 bytes reals
do i=1,6*31
print*,i,i*911856
read(10,REC=i) x
print*,minval(x),maxval(x)
enddo
end
In ferret I did (avoiding any issues with the 6 variables and 31 days I just use 186 “time” steps):
yes? define axis/x=1:227964:1 xa
yes? define axis/t=1:186:1 xt
yes? define grid/x=xa/t=xt xgr
yes? file/var=xx/grid=xgr/format=stream/type=r4/swap GB_1km_197101.bin
yes? show data
currently SET data sets:
1> ./GB_1km_197101.bin (default)
name title I J K L M N
XX XX 1:227964 ... ... 1:186 ... ...
yes? stats xx[l=1]
XX
X: NaN to NaN
Y: N/A
Z: N/A
T: NaN
E: N/A
F: N/A
DATA SET: ./GB_1km_197101.bin
Total # of data points: 227964 (227964*1*1*1*1*1)
# flagged as bad data: 0
Minimum value: -3.6893E+19
Maximum value: 3.6893E+19
Mean value: 2.3629E+16 (unweighted average)
Standard deviation: 1.0572E+19
Min and Max should be:
4.38666296 51.8558731
From: Karl Smith - NOAA Affiliate [mailto:karl.smith@xxxxxxxx]
Sent: 02 September 2013 08:52
To: Kaduk, Joerg (Dr.)
Cc: ferret_users@xxxxxxxx
Subject: Re: [ferret_users] Data reading
Hi Joerg,
The code at the line given in the error message is encountered when Ferret is trying the determine the best format for listing out your data as character (string) values to the console. I am not sure why you are encountering this error (assuming that error message is accurate); maybe the magnitude of the data values, maybe the data was not interpreted correctly when read in. (Some Fortran binary data file formats have a count value at the beginning that was either 4 bytes or 8 bytes, depending on version of gfortran used.)
But you might be able to work around this problem by not listing any values to the console and instead just saving all the data in a netcdf file after you read it into Ferret. So after reading in the data, issue the Ferret command:
LIST /FORMAT=CDF /FILE=GB_1km_197101.nc SD, LD, TA, Q, W, P
Since this writes the data using a completely different method, you should not encounter the error you mentioned. You can then cancel the data read in from the binary stream file, and use the netcdf file you just created for the data. You probably want to use the Ferret "statistics" command on the dataset variables to make sure values appear to be interpreted correctly. Then maybe listing some of the values. If the error shows up again, you might try the ncdump command (from the shell prompt - not a ferret command). Probably first just list the headers "ncdump -h", then maybe a full dump, to see if the data values appear to be correct.
Karl
On Sun, Sep 1, 2013 at 4:50 PM, Kaduk, Joerg (Dr.) <jk61@xxxxxxxxxxxxxxx> wrote:
Hello,
I have a reading problem:
I have a stream big endian r*4 file with 6 variables on 227964 land points for 31 days.
The data are ordered such that there are first all variables of day 1, then of day 2, …
I tried the below, but ferret crashed (see below) – with or without /swap.
This is:
FERRET v6.84
Linux 2.6.32-220.23.1.el6.x86_64 64-bit - 08/13/13
I checked the file size and it fits with 4*6*227964*31
I could write fortran code to convert to netcdf, but it would be nice if I could do this via ferret….
Any help greatly appreciated.
Cheers,
Joerg
yes? define axis/x=1:227964:1 xa
yes? define axis/t=1:31:1 xt
yes? define grid/x=xa/t=xt xgr
yes? file/var=sd,ld,ta,q,w,p/grid=xgr/format=stream/type=r4/swap GB_1km_197101.bin
yes? show data
currently SET data sets:
1> ./GB_1km_197101.bin (default)
name title I J K L M N
SD SD 1:227964 ... ... 1:31 ... ...
LD LD 1:227964 ... ... 1:31 ... ...
TA TA 1:227964 ... ... 1:31 ... ...
Q Q 1:227964 ... ... 1:31 ... ...
W W 1:227964 ... ... 1:31 ... ...
P P 1:227964 ... ... 1:31 ... ...
yes? SET MEMORY/SIZE=1000
Cached data cleared from memory
yes? list sd[i=1:100,l=1]
VARIABLE : SD
FILENAME : GB_1km_197101.bin
SUBSET : 100 points (X)
T : NaN
At line 72 of file var_data_fmt.F
Fortran runtime error: End of record
Jörg Kaduk
Senior lecturer
http://www.geog.le.ac.uk/staff/jk61/
Department of Geography, University of Leicester, University Road, Leicester LE1 7RH, UK
Elite Without Being Elitist
Times Higher Awards Winner 2007, 2008, 2009, 2010, 2011, 2012, 2013
--
Karl M. Smith, Ph.D.
JISAO Univ. Wash. and TMAP/PMEL NOAA
"The contents of this message are mine personally and do
not necessarily reflect any position of the Government
or the National Oceanic and Atmospheric Administration."