[Thread Prev][Thread Next][Index]

Re: [ferret_users] Problems with netcdf output and zaxreplace



Try making a plot of the variable you've defined.  That will load it into memory, doing all the transformations.

   shade stc_trans_rho_1025



On 6/4/2020 8:22 AM, Giorgio Graffino wrote:
Thanks Ansley for your suggestions.

Unfortunately I'm running Ferret on a cluster, because data are too big to be saved on my local computer, so I don't know whether I miht install a newer version of Ferret by myself. I'll ask to my cluster support. Meanwhile, I tried to increase the amount of allocated memory and removed all subsetting, but the process got killed by the system.

Sorry, the script I sent also included other operations. I'm resuming here the most relevant lines:

!----- MERIDIONAL TRANSPORT ON DENSITY ------------
define axis/z=1020:1030:0.5/unit="density"/depth dens_lev
let ty_trans_pacific_rho = zaxreplace(ty_trans_pacific,dens[g=ty_trans],z[gz=dens_lev])

!--- COMPUTE TIME AVERAGE ON 25 SIGMA LEVEL -------
let ty_trans_pacific_rho_time = ty_trans_pacific_rho[l=@ave]
let ty_trans_pacific_rho_lats = ty_trans_pacific_rho_time[y=35n:35s]
let ty_trans_pacific_rho_lons = ty_trans_pacific_rho_lats[x=110e:70w]
let ty_trans_pacific_rho_1025 = ty_trans_pacific_rho_lons[z=1025]

!---------- RESHAPE OUTPUT FIELDS -----------------
let/title="Meridional transport (Sv) on the 1025 kg/m3 isopycnal surface" stc_trans_rho_1025 = reshape(ty_trans_pacific_rho_1025,x[gx=ty_trans]+y[gy=ty_trans])

!------------ OUTPUT FILE -------------------------
save/file=STC_sigma_1PctTo2X_($model)_pacific.nc/clobber stc_trans_rho_1025

The error can't be related to append, because I'm saving only one variable into that file.  Other variables are saved into another file, but I have no problem with that.

Cheers,
Giorgio

    ----Messaggio originale----
    Da: ansley.b.manke@xxxxxxxx
    Data: 4-giu-2020 3.11
    A: <ferret_users@xxxxxxxx>
    Ogg: Re: [ferret_users] Problems with netcdf output and zaxreplace

    Hi Giorgio,
    When you can, try to update to a newer version of pyferret or ferret.
    Starting with Ferret v7.2
    <https://ferret.pmel.noaa.gov/Ferret/documentation/release_notes/version-7-2-release-notes>,

    there is much better memory management.  With your version of
    Ferret you
    can try "set memory/size="  to give it a larger  working memory
    buffer.

    For the errors you're seeing when trying to append to a file, the
    thing
    to know is that the grid of the variable you are appending needs to
    match what is in the file, except for time when appending in the time
    direction.  I can't quite see what is happening with the different
    grids
    and datasets in your script. What I would do to see what is
    happening,
    is at the point in your script where the error occurs, instead write
    that variable to a second file, and then compare the new file with
    the
    one you were trying to append to.  The error message is saying
    that the
    latitude coordinates don't match.

    Ansley

    On 6/3/2020 10:51 AM, Giorgio Graffino wrote:
    > Thanks Ashley about those useful tips.
    >
    > I'm now adapting my zaxreplace script for other applications (see
    > attached), but I have again issues on saving the output. I think
    the
    > problem is that the grid is too large for some models, and Ferret
    > complains by saying
    >
    > **ERROR: request exceeds memory setting: A negative number of words
    > were requested.
    > *** NOTE: The current grid is most likely too large
    >
    > So I'm subsetting the variable to select only the region I need,
    and
    > the computation works. But when I'm saving the generated field into
    > netcdf at line 52, Ferret comes out with the good ol' error
    >
    > **TMAP ERR: error in line definition
    >             file coords dont match variable coords on axis
    LAT77_203
    >
    > It doesn't change whether I'm using RESHAPE or not. I also tried to
    > adapt my RESHAPE call according to what I read here
    >
    (https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/XPRESSIONS),

    > but with no luck. Can you please help me again about this?
    >
    > Cheers,
    > Giorgio
    >
    >     ----Messaggio originale----
    >     Da: ansley.b.manke@xxxxxxxx
    >     Data: 20-mag-2020 18.52
    >     A: "Giorgio Graffino"<g.graffino@xxxxxx>
    >     Cc: <ferret_users@xxxxxxxx>
    >     Ogg: Re: [ferret_users] Problems with netcdf output and
    zaxreplace
    >
    >     Hi,
    >
    >     You could pass into the script the number of models
    >
    >     > ferret -script scriptname.jnl 7
    >
    >     The script then does a repeat loop that would start like this.
    >
    >        let nmodel = ($1)
    >        repeat/range=1:($1)/name=r (\
    >        use /home/data/model`r`.nc;\
    >
    >     ...
    >
    >     There are details on the grave-accent evaluation
    >
    <https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/EMBEDDED-XPRESSIONS#_VPINDEXENTRY_562>that

    >     give you some flexibility.  For instance if the models are
    named
    >     model01.nc, model02.nc, ...  use the "precision" specifier
    to ask
    >     for the value to be replaced with zero-filled numbers, here
    2-digits.
    >
    >        let nmodel = ($1)
    >        repeat/range=1:`nmodel`/name=r (\
    >        use /home/data/model`r,p=z2`.nc;
    >
    >
    >     Or, you could use a unix foreach loop and write your script to
    >     operate on just one model if that makes sense for what you are
    >     doing. Here I've made a script myscript.jnl with:
    >
    >     ! myscript.jnl
    >     echo ($1)
    >
    >
    >     This is csh, other shells will have other syntax.
    >
    >     /home/users/ansley> set flist = 'model_run_7.nc
    >     ocean_model_run5.nc http://address/climate.nc'
    >     /home/users/ansley> foreach fname ($flist)
    >     foreach? ferret -script myscript.jnl $fname
    >     foreach? end
    >     model_run_7.nc
    >     ocean_model_run5.nc
    > http://address/data/climate.nc
    >
    >
    >     On 5/18/2020 10:47 AM, Giorgio Graffino wrote:
    >>     Hi Ashley,
    >>     thanks for your suggestion.
    >>
    >>     I preferred to implement the other method because it
    involved a
    >>     single Ferret call, but I haven't thought about passing all
    >>     variables in that way. You are suggesting something like
    >>
    >>     ferret -script scriptname.jnl model1 model2 model3
    >>
    >>     which would be great to have, but I should find a way to loop
    >>     across all models from inside the jnl script. Now my jnl
    script
    >>     includes
    >>
    >>     define symbol model = ($01)
    >>
    >>     to read the argument, because I'm parsing only one argument
    at a
    >>     time. How can I loop over all models from inside the jnl
    script?
    >>
    >>     Cheers,
    >>     Giorgio
    >>
    >>
    >>         ----Messaggio originale----
    >>         Da: ansley.b.manke@xxxxxxxx
    >>         Data: 12-mag-2020 19.55
    >>         A: <ferret_users@xxxxxxxx>
    >>         Ogg: Re: [ferret_users] Problems with netcdf output and
    >>         zaxreplace
    >>
    >>         Hi Giorgio,
    >>
    >>         Another method for sending Ferret/PyFerret a set of
    commands
    >>         is to use the -script option on startup.
    >>
    >>         > pyferret -script my_script.jnl [arguments]
    >>
    >>         This is generally more flexible than piping commands.  The
    >>         startup option -batch for Ferret or -nodisplay for
    PyFerret
    >>         are also useful when you don't want to have graphics
    windows
    >>         display.  Note that -script must appear last among command
    >>         line switches so that the script arguments will be parsed
    >>         correctly.
    >>
    >>         Unix command-line switches
    >>
    <https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/introduction/GETTING-STARTED#_VPID_8>

    >>
    >>
    >>
    >>         On 5/9/2020 3:19 AM, Giorgio Graffino wrote:
    >>>         Hi Ryo,
    >>>         thanks to your "quote" suggestion I could successfully
    run
    >>>         my bash script.
    >>>
    >>>         Cheers,
    >>>         Giorgio
    >>>
    >>>             ----Messaggio originale----
    >>>             Da: furue@xxxxxxxxxx
    >>>             Data: 28-apr-2020 14.28
    >>>             A: "Giorgio Graffino" <g.graffino@xxxxxx>
    >>>             Cc: "Ferret" <ferret_users@xxxxxxxx>
    >>>             Ogg: Re: [ferret_users] Problems with netcdf
    output and
    >>>             zaxreplace
    >>>
    >>>             Hi Giorgio,
    >>>
    >>>             On Tue, Apr 28, 2020 at 7:09 PM Giorgio Graffino <
    >>>             g.graffino@xxxxxx <invalidurl.gif>> wrote:
    >>>
    >>>
    >>>                 !-> REPEAT: I=1
    >>>                 **ERROR: command syntax: II = ???
    >>>                 define symbol ii
    >>>
    >>>
    >>>             Compare this line with the one in the original shell
    >>>             script:
    >>>
    >>>             ferret << STOP
    >>>             . . .
    >>>             define symbol ii `i`
    >>>             . . .
    >>>             STOP
    >>>
    >>>             Where did `i`  go?
    >>>
    >>>             In the shell script as well as in the ferret
    script, the
    >>>             backquote construct is evaluated.  In the above shell
    >>>             script, the shell tries to run a command named
    "i".  It
    >>>             fails and is replaced with an empty string. I
    >>>             suppose you saw this error message on your screen
    >>>
    >>>             . . . i: command not found
    >>>
    >>>             To prevent the shell from interpreting the backquote
    >>>             construct, you "quote" the here-document by
    >>>
    >>>             ferret <<'STOP'
    >>>             . . .
    >>>             STOP
    >>>
    >>>             Ryo
    >>>             --------------------
    >>>             To see what's going on, run this kind of test
    shell script
    >>>
    >>>             echo date
    >>>             echo `date`
    >>>             echo "abc `date` def"
    >>>             echo 'abc `date` def'
    >>>
    >>>             cat <<EOF
    >>>             date
    >>>             `date`
    >>>             EOF
    >>>
    >>>             cat <<'EOF'
    >>>             date
    >>>             `date`
    >>>             EOF
    >>>
    >>>
    >>>
    >>         --
    >>         Ansley Manke
    >>         Science Data Integration Group
    >>         NOAA Pacific Marine Environmental Laboratory
    >>         7600 Sand Point Way NE
    >>         Seattle WA 98115
    >>
    >>         I am currently teleworking and am available Tue-Wed-Thu.
    >>
    >>
    >>
    >     --
    >     Ansley Manke
    >     Science Data Integration Group
    >     NOAA Pacific Marine Environmental Laboratory
    >     7600 Sand Point Way NE
    >     Seattle WA 98115
    >
    >     I am currently teleworking and am available Tue-Wed-Thu.
    >
    >
    >

-- Ansley Manke
    Science Data Integration Group
    NOAA Pacific Marine Environmental Laboratory
    7600 Sand Point Way NE
    Seattle WA 98115

    I am currently teleworking and am available Tue-Wed-Thu.




--
Ansley Manke
Science Data Integration Group
NOAA Pacific Marine Environmental Laboratory
7600 Sand Point Way NE
Seattle WA 98115

I am currently teleworking and am available Tue-Wed-Thu.


[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement