Re: [ferret_users] Simultaneous ferret execs with -gif lead to "file exists" error

[Andrew Wittenberg - NOAA Federal <andrew.wittenberg@xxxxxxxx>]

Hi Bill,

I'll let one of the Ferret developers answer the question about file
naming.  But a quick workaround would be to have your Python script
create, and then cd into, a temporary directory named using the
process ID.  Run your script inside that directory, and then move the
resulting image to your desired path.

Andrew

On Tue, Feb 12, 2013 at 7:47 PM, Gustafson, William I
<william.gustafson@xxxxxxxx> wrote:
> I have written a Python script to wrap around a Ferret script so that I
> can simultaneously generate lots of panels for an animation. The Python
> script uses MPI to distribute lots (order 100's) of Ferret sessions across
> a Linux cluster. The problem is that it almost works, but not quite. A
> small percentage of the resulting gif files are not generated because
> Ferret thinks the output file already exists. I get errors like:
>
>         **ERROR: .gif: File exists
>         gdeactivatews()    3 GKS not in proper state: GKS shall be in the state
> WSAC
>         gactivatews()    6 GKS not in proper state: GKS shall be either in the
>         state WSOP or in the state WSAC
>         ginqwsconntype()    7 GKS not in proper state: GKS shall be in one of the
>         states WSOP, WSAC or SGOP
>         gsetwswindow()    7 GKS not in proper state: GKS shall be in one of the
>         states WSOP, WSAC or SGOP
>         gescsetdcsize()    7 GKS not in proper state: GKS shall be in one of the
>         states WSOP, WSAC or SGOP
>         gupdatews()    7 GKS not in proper state: GKS shall be in one of the
>         states WSOP, WSAC or SGOP
>         STOP -script mode, ERROR RUNNING SCRIPT
>         Cached data cleared from memory
>         **ERROR: required program command has not been given: graphical output
>         device isnt ready
>         POLYGON/nolab/noaxes/line=1 {0,0,1,1,0},{0,1,1,0,0}
>         Command file, command group, or REPEAT execution aborted
>
> The Ferret script uses the command "frame/file=anim`aa`.gif" to write the
> file, and Ferret is executed using the "-gif -script <scriptname>" command
> line options. I have verified that every process is using unique filenames
> so there should be no overlap problem.
>
> Has anybody else run into this sort of error? Does Ferret's frame command
> use a generic name and then rename the file to the final output name? If
> so, that could lead to this sort of behavior. Any suggestions on how to
> get around this?
>
> If it helps, here is the relevant parts of the Ferret script (note that
> plot_bp is a 2nd script that does the actual plot commands):
>
>         define symbol start = $1
>         define symbol end   = $2
>         let aa=($start)
>         repeat/l=($start):($end) ( ;\
>           go plot_bp ;\
>           frame/file="anim/anim`aa`.gif" ;\
>           let aa=`aa+1` ;\
>         )
>
>
> And, here is the main portion of the Python script:
>
>         comm = MPI.COMM_WORLD
>         nproc = comm.Get_size()
>         myrank = comm.Get_rank()
>
>         script = "anim_auto.jnl" # Ferret script name
>         t1     = 0         # 0-based start index for input file time
>         t2     = 100       # 0-based end index for input file time
>
>         #
>         # Have each task call Ferret for its set of times...
>         #
>         t1me,t2me = distribute_times(t1,t2,myrank,nproc,comm) # Returns times for
> this task
>         ntme = t2me-t1me+1
>
>         for k in xrange(t1me,t2me+1):
>             cmd  = "csh -c \"source ~/ferret_paths; ferret -gif -script
> "+script+" %d %d\""%(k+1,k+1)
>             os.system(cmd)
>
> Also, if it would matter, I am running on a cluster that uses a Lustre
> file system. I have had some trouble in the past with some programs that
> expect to be able to do file locking, which Lustre doesn't like. But, I do
> not think that is the case for Ferret.
>
> Thanks,
>
>    Bill
>
>
>
> ___________________________________________________________________
>
> William I. Gustafson Jr., Ph.D.
> Scientist
> ATMOSPHERIC SCIENCES AND GLOBAL CHANGE DIVISION
>
> Pacific Northwest National Laboratory
> P.O. 999, MSIN K9-30
> Richland, WA  99352
> Tel: 509-372-6110
> William.Gustafson@xxxxxxxx
> http://www.pnnl.gov/atmospheric/staff/staff_info.asp?staff_num=5716
> http://www.researcherid.com/rid/A-7732-2008
>
>