Hi Karl,
thank you very much. So it is not a simple internal python issue
or something like this. I also did not notice another similar
limitation. Shortly before the limit in teh string number shows
up, an internal limit is met:
**TMAP ERR: limit on number of axes has been reached
Meanwhile I am working "piecewise".
Best,
Martin
Am 03.06.2019 um 18:41 schrieb Karl
Smith - NOAA Affiliate:
Hi Martin,
I was starting to write you about how Ferret/PyFerret reads
the results of a command (the C function fopen) when I noticed
that there is a hard limit of 2000 strings per string variable
still being imposed in the code without proper error
handling. I am fairly sure this limitation should no longer
be imposed, but will need to be investigated. And, obviously,
if it does still have to be imposed, the limit needs to be
increased and given proper error handling.
Best,
Karl
Dear
all,
I am trying to glue a large amount of files using the
aggregate
capability of ferret. The problem is the file list itself. The
files are
ordered by a 4-digit number. So
ls -l wind_0*.nc
gives me 1000 file names,
ls -l wind_[0-2]*.nc
results in 3000 file names. All works well with up to 2000
file names.
In ferret:
yes? let file_list = SPAWN("ls -1 wind_0*nc")
yes? list file_list
gives the correct list and the aggregation works great as it
should.
yes? let file_list = SPAWN("ls -1 wind_[0-2]*nc")
yes? list file_list
results in huge traceback and ferret quits.
yes? list file_list
*** Error in
`/sw/data/python/miniconda/OS_42.3/2.7.15/bin/python':
free(): invalid pointer: 0x00007fcfbc746698 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x740ef)[0x7fcfbc41b0ef]
/lib64/libc.so.6(+0x79646)[0x7fcfbc420646]
/lib64/libc.so.6(+0x7a393)[0x7fcfbc421393]
/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/libpyferret.so(FerMem_Free+0x1a)[0x7fcfb622ee83]
...
/sw/data/python/miniconda/OS_42.3/2.7.15/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7d14)[0x7fcfbd1b2ea4]
... several 100 similar lines from many other libraries in the
miniconda
based python including a memory map and finally
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/__init__.py",
line 482, in init
result = run()
File
"/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/__init__.py",
line 887, in run
retval = libpyferret._run(str_command)
RuntimeError: etc
I have no idea, which limit or ulimit, (but it is not the
stacksize)
could be violated. I have no idea how to produce this list in
python
directly, to check if this a python or ferret issue.
in _init_.py it crashs here:
readline.set_completer_delims(FERRET_READLINE_DELIMS)
# the actual Ferret function call
retval = libpyferret._run(str_command)
Is there eventually a limit in ferret itself?
Any hints would be very welcome.
Best,
Martin
--
Karl M. Smith, Ph.D.
JISAO Univ. Wash. and PMEL NOAA
"The contents of this message are mine personally and do
not necessarily reflect any position of the Government
or the National Oceanic and Atmospheric Administration."
|