Thank you! That is very helpful in understanding the system behavior.
If I understand you correctly:
1) The cache.obj ‘s main purpose is for quick reference on requests; if it is in cache.obj, the system looks for it in /usr/local/tomcat/webapps/las/output/ and returns it if it exists, saving lots of time
2) Cache.obj does not actively have anything to do with keeping the cache size under the constraints stated in projectserver.xml
3) The number of files and the size of the files in /output are what determine when an automatic purge happens; with our settings, at 1500 files and 1G, respectively
If the above is true, then we are puzzled – our current /usr/local/tomcat/webapps/las/output/ has 10,750 files, and ~18G in it. It has filled the disk before, and started producing files of 0 bytes – and causing errors (LAS returns errors complaining about map scales) which is how we caught it in the first place.
Does the purge only happen upon stop/restart of Tomcat? Or is something else going on? We would like to be able to keep the cache disk use dynamically culled, to prevent disk fill-up and errors. Is there a way to do this?
From: Roland Schweitzer - NOAA Affiliate [mailto:roland.schweitzer@xxxxxxxx]
Sent: Thursday, May 09, 2013 2:18 PM
To: Shelton, Kacie E (398C)
Cc: las_users@xxxxxxxx; Sumagaysay, Rosanna M (3980-Affiliate)
Subject: Re: [las_users] Questions on cleaning & maintaining the cache manually
Sorry for the delay in answering. I answered this in my head, but never put fingers to keyboard so here goes...
On Thu, May 2, 2013 at 6:38 PM, Shelton, Kacie E (398C) <kacie.e.shelton@xxxxxxxxxxxx> wrote:
We have not installed/setup the user interface the web maintenance, and have a few questions on how the productserver.xml, the cache.obj, and the /usr/local/tomcat/webapps/las/output/ directory interact with each other, and how we can manage the cache. Recently our disk was filled by files in /output/ , and we are looking to understand the processes and avoid disk-fill in the future.
Our productserver.xml has this setup:
<cache size="1500" bytes="1024Mb" file="/usr/local/tomcat/webapps/las/output/cache.obj"/>
1) Does the cache size = 1500 refer to the number of lines in the cache.obj file, or the number of files currently in the /webapps/las/output/ directory?
The limit is on the number of files in the cache when the server is running. The cache.obj file is only written when the server stops and is read with the server restarts.
2) The files in the /output/ directory that start with the same ID - for example,
-- Are these files considered one file by the cache size, or four files?
That's 4 different files as far as the cache is concerned.
3) Does the bytes=1024Mb indicates the files size of cache.obj or the total size of the directory /usr/local/tomcat/webapps/las/output/ ?
Total size of the files in the output directory.
Manual Cache clearing:
4) Can we clean the cache by doing:
Delete all files in /usr/local/tomcat/webapps/las/output/ (Or all but cache.obj?)
If you remove the files from the output directory you should also remove the cache.obj file since it be read when the server restarts and would have the cache in memory pointing to a bunch of files that don't exist. Which doesn't matter in the end as you'll see in a moment.
What you have described will work and is certainly the safest way to clean the cache.
Automatic purge settings:
5) How can we check/confirm the number of files listed in cache.obj? It's difficult to read/parse the file.
That file is a direct serialization of the Cache Java object, but in the end it doesn't matter what's in that file except that it will help the system to keep the cache alive between server restarts. LAS checks two things when it is looking for a cache hit: 1) is the file in the memory Cache object and 2) does the file actually exist. If both are true for every file needed for a product, then the request is served by returning the response immediately pointing to the cached files. If the memory version of the Cache (which was read once from cache.obj if it exists last time the server started and then manged in memory from that point forward) says a file is in the output directory (the cache), but the file does not exist then that reference is kicked out of the memory cache and the systems goes about creating all the files needed to satisfy the request and if it is successful and making the product those files are entered into the memory cache so they can be used for the next time that request occurs.
6) Currently our file count in /usr/local/tomcat/webapps/las/output directory is: 10,735, way above file size=1500. We thought that cache.obj would purge older files, bringing it down to below 75% of 1500? Or is cache.obj only counting the ID’s, not the individual files? (as in question 2 -- We are not sure if the current cache.obj has in its list 10,735 files, due to having difficulty parsing it.)
Ok. So all of the above is the theory of how it works, but the reality is that if that Tomcat is started and the cache.obj file does not exist, the files in the output directory will just sit there orphaned from the system unless and until the exact same request comes in and they are refreshed and then get added to the memory cache and the maybe managed between restarts by a fresh copy of the cache.obj written the next time the Tomcat is stopped.
But, it's easier to manage that it sounds because in practice you can remove files from the cache (output directory) of a running system with impunity. If a request for a product needs that file and it's not on disk even though memory cache says is should be the system will remake it no problem.
Finally, there is code already checked in for the next release that will make LAS more actively manage the cache. You can specify how often the cache gets cleaned, and how old an output has to be before it is removed. The new code will also just populate the memory cache by reading the directory contents instead of relying on the cache.obj which might have been removed for some reason after system stopped, but before it was restarted.
Thanks very much,