[las_users] Problems with large number of open files at Opendap/THREDDS (including possible solution)

[Bastian Kern <bkern@xxxxxxxxxxxxxxxxx>]

Hi all,

with a ferret script accessing LAS data via Opendap urls, I discovered a 
problem, if a loop reads many datapoints. The error is: "Too many open files" 
and the Tomcat Server stops. Below some lines of the catalina.out file:

> Apr 9, 2009 12:32:35 PM org.apache.tomcat.util.net.PoolTcpEndpoint
> acceptSocket SEVERE: Endpoint
> ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8080] ignored exception:
> java.net.SocketException: Too many open files java.net.SocketException: Too
> many open files
>         at java.net.PlainSocketImpl.socketAccept(Native Method)
>         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>         at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>         at java.net.ServerSocket.accept(ServerSocket.java:421)
>         at
> org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultS
>erverSocketFactory.java:61) at
> org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.jav
>a:408) at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerW
>orkerThread.java:71) at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.ja
>va:689) at java.lang.Thread.run(Thread.java:619)
> Apr 9, 2009 12:32:35 PM org.apache.tomcat.util.net.PoolTcpEndpoint
> acceptSocket WARNING: Reinitializing ServerSocket
> Apr 9, 2009 12:32:37 PM org.apache.tomcat.util.net.PoolTcpEndpoint
> acceptSocket SEVERE: Endpoint
> ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8080] ignored exception:
> java.net.SocketException: Too many open files java.net.SocketException: Too
> many open files
>         at java.net.PlainSocketImpl.socketAccept(Native Method)
>         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>         at java.net.ServerSocket.implAccept(ServerSocket.java:453)
>         at java.net.ServerSocket.accept(ServerSocket.java:421)
>         at
> org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultS
>erverSocketFactory.java:61) at
> org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.jav
>a:408) at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerW
>orkerThread.java:71) at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.ja
>va:689) at java.lang.Thread.run(Thread.java:619)
> Apr 9, 2009 12:32:37 PM org.apache.tomcat.util.net.PoolTcpEndpoint
> acceptSocket WARNING: Reinitializing ServerSocket

Possible solution:

Edit the FerretIOServiceProvider.java file at: 
$LASDIR/JavaSource/gov/noaa/pmel/tmap/iosp/, so that opend files will be 
closed.
I added a variable

> private RandomAccessFile raf;

to the public FerretIOServiceProvider class and changed close() and 
open(RandomAccessFile raf, NetcdfFile ncfile, CancelTask cancelTask):

>     public void close() throws IOException {
>         // TODO Auto-generated method stub
>         raf.close();                                // NEW !!
>     }

>     public void open(RandomAccessFile raf, NetcdfFile ncfile,
>             CancelTask cancelTask) throws IOException {
>         log.debug("Opening " + raf.getLocation());
>         this.raf = raf;                             // NEW !!
>         raf.seek(0);
>         StringReader sr;
>         try {
>             byte[] b = new byte[(int)raf.length()];
>             raf.read(b);
>             sr = new StringReader(new String(b));
>         } catch (IOException e) {
>             log.debug("IO Exception reading the random access file.");
>             throw e;
>         }
> [...]

Best regards,
Bastian

-- 
---------------------------------------------------------------
Bastian Kern
Max Planck Institute for Chemistry
> Joh.-J. Becher Weg 27
Joh.-J. Becher Weg 27
55128 Mainz                              phone:++49-6131-305452
Germany                          mailto:bkern@xxxxxxxxxxxxxxxxx
---------------------------------------------------------------