Re: [ferret_users] Problems with netcdf output and zaxreplace

[Giorgio Graffino <g.graffino@xxxxxx>]

Thanks Ansley for your suggestions.

Unfortunately I'm running Ferret on a cluster, because data are too big to be saved on my local computer, so I don't know whether I miht install a newer version of Ferret by myself. I'll ask to my cluster support. Meanwhile, I tried to increase the amount of allocated memory and removed all subsetting, but the process got killed by the system.

Sorry, the script I sent also included other operations. I'm resuming here the most relevant lines:

!----- MERIDIONAL TRANSPORT ON DENSITY ------------

define axis/z=1020:1030:0.5/unit="density"/depth dens_lev 

let ty_trans_pacific_rho = zaxreplace(ty_trans_pacific,dens[g=ty_trans],z[gz=dens_lev])

!--- COMPUTE TIME AVERAGE ON 25 SIGMA LEVEL -------

let ty_trans_pacific_rho_time = ty_trans_pacific_rho[l=@ave]

let ty_trans_pacific_rho_lats = ty_trans_pacific_rho_time[y=35n:35s]

let ty_trans_pacific_rho_lons = ty_trans_pacific_rho_lats[x=110e:70w]

let ty_trans_pacific_rho_1025 = ty_trans_pacific_rho_lons[z=1025]

!---------- RESHAPE OUTPUT FIELDS -----------------

let/title="Meridional transport (Sv) on the 1025 kg/m3 isopycnal surface" stc_trans_rho_1025 = reshape(ty_trans_pacific_rho_1025,x[gx=ty_trans]+y[gy=ty_trans])

!------------ OUTPUT FILE -------------------------

save/file=STC_sigma_1PctTo2X_($model)_pacific.nc/clobber stc_trans_rho_1025

The error can't be related to append, because I'm saving only one variable into that file.  Other variables are saved into another file, but I have no problem with that.

Cheers,

Giorgio

----Messaggio originale----
Da: ansley.b.manke@xxxxxxxx
Data: 4-giu-2020 3.11
A: <ferret_users@xxxxxxxx>
Ogg: Re: [ferret_users] Problems with netcdf output and zaxreplace

Hi Giorgio,
When you can, try to update to a newer version of pyferret or ferret.
Starting with Ferret v7.2
<https://ferret.pmel.noaa.gov/Ferret/documentation/release_notes/version-7-2-release-notes>,
there is much better memory management.  With your version of Ferret you
can try "set memory/size="  to give it a larger  working memory buffer.

For the errors you're seeing when trying to append to a file, the thing
to know is that the grid of the variable you are appending needs to
match what is in the file, except for time when appending in the time
direction.  I can't quite see what is happening with the different grids
and datasets in your script. What I would do to see what is happening,
is at the point in your script where the error occurs, instead write
that variable to a second file, and then compare the new file with the
one you were trying to append to.  The error message is saying that the
latitude coordinates don't match.

Ansley

On 6/3/2020 10:51 AM, Giorgio Graffino wrote:
> Thanks Ashley about those useful tips.
>
> I'm now adapting my zaxreplace script for other applications (see
> attached), but I have again issues on saving the output. I think the
> problem is that the grid is too large for some models, and Ferret
> complains by saying
>
> **ERROR: request exceeds memory setting: A negative number of words
> were requested.
> *** NOTE: The current grid is most likely too large
>
> So I'm subsetting the variable to select only the region I need, and
> the computation works. But when I'm saving the generated field into
> netcdf at line 52, Ferret comes out with the good ol' error
>
> **TMAP ERR: error in line definition
>             file coords dont match variable coords on axis LAT77_203
>
> It doesn't change whether I'm using RESHAPE or not. I also tried to
> adapt my RESHAPE call according to what I read here
> (https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/XPRESSIONS),
> but with no luck. Can you please help me again about this?
>
> Cheers,
> Giorgio
>
>     ----Messaggio originale----
>     Da: ansley.b.manke@xxxxxxxx
>     Data: 20-mag-2020 18.52
>     A: "Giorgio Graffino"<g.graffino@xxxxxx>
>     Cc: <ferret_users@xxxxxxxx>
>     Ogg: Re: [ferret_users] Problems with netcdf output and zaxreplace
>
>     Hi,
>
>     You could pass into the script the number of models
>
>     > ferret -script scriptname.jnl 7
>
>     The script then does a repeat loop that would start like this.
>
>        let nmodel = ($1)
>        repeat/range=1:($1)/name=r (\
>        use /home/data/model`r`.nc;\
>
>     ...
>
>     There are details on the grave-accent evaluation
>     <https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/EMBEDDED-XPRESSIONS#_VPINDEXENTRY_562>that
>     give you some flexibility.  For instance if the models are named
>     model01.nc, model02.nc, ...  use the "precision" specifier to ask
>     for the value to be replaced with zero-filled numbers, here 2-digits.
>
>        let nmodel = ($1)
>        repeat/range=1:`nmodel`/name=r (\
>        use /home/data/model`r,p=z2`.nc;
>
>
>     Or, you could use a unix foreach loop and write your script to
>     operate on just one model if that makes sense for what you are
>     doing. Here I've made a script myscript.jnl with:
>
>     ! myscript.jnl
>     echo ($1)
>
>
>     This is csh, other shells will have other syntax.
>
>     /home/users/ansley> set flist = 'model_run_7.nc
>     ocean_model_run5.nc http://address/climate.nc'
>     /home/users/ansley> foreach fname ($flist)
>     foreach? ferret -script myscript.jnl $fname
>     foreach? end
>     model_run_7.nc
>     ocean_model_run5.nc
>     http://address/data/climate.nc
>
>
>     On 5/18/2020 10:47 AM, Giorgio Graffino wrote:
>>     Hi Ashley,
>>     thanks for your suggestion.
>>
>>     I preferred to implement the other method because it involved a
>>     single Ferret call, but I haven't thought about passing all
>>     variables in that way. You are suggesting something like
>>
>>     ferret -script scriptname.jnl model1 model2 model3
>>
>>     which would be great to have, but I should find a way to loop
>>     across all models from inside the jnl script. Now my jnl script
>>     includes
>>
>>     define symbol model = ($01)
>>
>>     to read the argument, because I'm parsing only one argument at a
>>     time. How can I loop over all models from inside the jnl script?
>>
>>     Cheers,
>>     Giorgio
>>
>>
>>         ----Messaggio originale----
>>         Da: ansley.b.manke@xxxxxxxx
>>         Data: 12-mag-2020 19.55
>>         A: <ferret_users@xxxxxxxx>
>>         Ogg: Re: [ferret_users] Problems with netcdf output and
>>         zaxreplace
>>
>>         Hi Giorgio,
>>
>>         Another method for sending Ferret/PyFerret a set of commands
>>         is to use the -script option on startup.
>>
>>         > pyferret -script my_script.jnl  [arguments]
>>
>>         This is generally more flexible than piping commands.  The
>>         startup option -batch for Ferret or -nodisplay for PyFerret
>>         are also useful when you don't want to have graphics windows
>>         display.  Note that -script must appear last among command
>>         line switches so that the script arguments will be parsed
>>         correctly.
>>
>>         Unix command-line switches
>>         <https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/introduction/GETTING-STARTED#_VPID_8>
>>
>>
>>
>>         On 5/9/2020 3:19 AM, Giorgio Graffino wrote:
>>>         Hi Ryo,
>>>         thanks to your "quote" suggestion I could successfully run
>>>         my bash script.
>>>
>>>         Cheers,
>>>         Giorgio
>>>
>>>             ----Messaggio originale----
>>>             Da: furue@xxxxxxxxxx
>>>             Data: 28-apr-2020 14.28
>>>             A: "Giorgio Graffino" <g.graffino@xxxxxx>
>>>             Cc: "Ferret" <ferret_users@xxxxxxxx>
>>>             Ogg: Re: [ferret_users] Problems with netcdf output and
>>>             zaxreplace
>>>
>>>             Hi Giorgio,
>>>
>>>             On Tue, Apr 28, 2020 at 7:09 PM Giorgio Graffino <
>>>             g.graffino@xxxxxx <invalidurl.gif>> wrote:
>>>
>>>
>>>                 !-> REPEAT: I=1
>>>                 **ERROR: command syntax: II = ???
>>>                 define symbol ii
>>>
>>>
>>>             Compare this line with the one in the original shell
>>>             script:
>>>
>>>             ferret << STOP
>>>             . . .
>>>             define symbol ii `i`
>>>             . . .
>>>             STOP
>>>
>>>             Where did `i`  go?
>>>
>>>             In the shell script as well as in the ferret script, the
>>>             backquote construct is evaluated.  In the above shell
>>>             script, the shell tries to run a command named "i".  It
>>>             fails and is replaced with an empty string. I
>>>             suppose you saw this error message on your screen
>>>
>>>             . . . i: command not found
>>>
>>>             To prevent the shell from interpreting the backquote
>>>             construct, you "quote" the here-document by
>>>
>>>             ferret <<'STOP'
>>>             . . .
>>>             STOP
>>>
>>>             Ryo
>>>             --------------------
>>>             To see what's going on, run this kind of test shell script
>>>
>>>             echo date
>>>             echo `date`
>>>             echo "abc `date` def"
>>>             echo 'abc `date` def'
>>>
>>>             cat <<EOF
>>>             date
>>>             `date`
>>>             EOF
>>>
>>>             cat <<'EOF'
>>>             date
>>>             `date`
>>>             EOF
>>>
>>>
>>>
>>         --
>>         Ansley Manke
>>         Science Data Integration Group
>>         NOAA Pacific Marine Environmental Laboratory
>>         7600 Sand Point Way NE
>>         Seattle WA 98115
>>
>>         I am currently teleworking and am available Tue-Wed-Thu.
>>
>>
>>
>     --
>     Ansley Manke
>     Science Data Integration Group
>     NOAA Pacific Marine Environmental Laboratory
>     7600 Sand Point Way NE
>     Seattle WA 98115
>
>     I am currently teleworking and am available Tue-Wed-Thu.
>
>
>

--
Ansley Manke
Science Data Integration Group
NOAA Pacific Marine Environmental Laboratory
7600 Sand Point Way NE
Seattle WA 98115

I am currently teleworking and am available Tue-Wed-Thu.