Re: [las_users] problem with smith_sandwell_topo_v8

On Tue, Oct 2, 2012 at 3:19 PM, Ansley Manke <ansley.b.manke@xxxxxxxx> wrote:

Hi-
I am not the right person in our group to help with this. The right person is probably Roland Schweitzer, who is away on travel this week, but I'll include him on this email.

Roland, the symtom that brought this up is the Ferret scripts trying to open a URL in order to make a land map:

   USE "http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc"

A good workaround is to ask the script to test whether it can open that dataset and if not, use a coarser land dataset. I've added such a test to the code which will be part of future LAS releases, and we could install that change on Sebastien's server, but there are other questions about the use of proxy's here, which I don't know about and which Roland may be able to address.

Ansley

On 10/2/2012 2:37 AM, Sebastien Lebosse wrote:

Hi Ansley and LAS users,

After the end of my holiday, I come to you. I had not yet found a solution to my problem. After consultation of our IT team, I can confirm that this access problem seems to be that my Ferret back-end service configuration or Ferret back-end service version (7.3) does not pass through our (or a) proxy (see below).

I have been searching around my .dodsrc or my FerretBackendConfig.xml but without success. Thanks in advance for your help.

Sébastien

Le 29/08/2012 15:15, Sebastien Lebosse a écrit :
Hi Ansley,

It's a funny misunderstanding. I am the technical support of the LAS Aviso. :-) We can continue this discussion...

LAS and F-TDS are in the same server but behind a proxy. As I said earlier, when "I try to run the corresponding LAS Ferret script from the command line as specified here http://ferret.pmel.noaa.gov/LAS/documentation/installer-documentation/errors-and-debugging/running-an-las-ferret-script-from-the-command-line/, there is no problem". In this case Ferret used the environmental variable http_proxy which is correctly set in .bashrc file.

However, it seems that the Ferret back-end service launched by the LAS Aviso does not pass through our proxy. How I can configure the proxy for the Ferret back-end service ?

As specified here http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/data-set-basics/ACCESS-TO-REMOTE-DATA-SETS-WITH-DODS/, I tried to configure the PROXY_SERVER variable in .dodsrc which is set in the FerretBackendConfig.xml but without success.
Perhaps there is something wrong in my .dodsrc or my FerretBackendConfig.xml - Perhaps this configuration doesn't work - I have perhaps forgotten something ?

Thank you once again for your support.
regards,
Sébastien

PS : you can see the attached files

---------------------------------- my .dodsrc --------------------------------------------
# DODS client configuration file. See the DODS
# users guide for information.
USE_CACHE=1
MAX_CACHE_SIZE=100
MAX_CACHED_OBJ=5
IGNORE_EXPIRES=0
CACHE_ROOT=/opt/atoll/tomcat-las-aviso/content/las/conf/server/dods/.dods_cache
DEFAULT_EXPIRES=86400
ALWAYS_VALIDATE=0
PROXY_SERVER=HTTP,http://proxy-prod.cls.fr:8080
-----------------------------------------------------------------------------------------------
---------------------- my FerretBackendConfig.xml --------------------------------
<application>
    <invoker base_dir="resources/ferret"
                 executable="/opt/atoll/products/ferret/bin/ferret"
                 time_limit="3600" template_dir="templates"
                 output_dir="/opt/atoll/tomcat-las-aviso/webapps/las/output"
                 http_base_url = "http://las.aviso.oceanobs.com/las/output"
                 iosp_script_dir="resources/iosp/scripts"
                 iosp_temp_dir="/opt/atoll/tomcat-las-aviso/content/las/conf/server/temp"
                 iosp_data_dir="/opt/atoll/tomcat-las-aviso/content/las/conf/server/data/dynamic"
                 las_tfds_data_dir="/opt/atoll/tomcat-las-aviso/content/las/conf/server/data"
                 iosp_base_dir="resources/iosp"
                 opendap_base_url = "http://las.aviso.oceanobs.com/thredds/dodsC/output"/>
    <cacher dataset="true" dataset_queue_size="10000" dataset_size="40"
        subset="true" subset_queue_size="10000" meta="true"
        meta_queue_size="10000" />
        <messages>
           <message type="startsWith" level="error">
              <key>PPL+ error</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR Sorry. This server cannot yet perform a time analysis and a month-of-the-year constraint together.</key>
              <text>
                  Sorry. This server cannot yet perform a time analysis and a month-of-the-year constraint together.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR "Desktop Access script not available for data on a Curvilinear grid</key>
              <text>
                Desktop Access script not available for data on a Curvilinear grid.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR The variables must have the same dimensions</key>
              <text>
                In order to compare multiple variables, they must all share the same dimensions.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR No t axis for variable</key>
              <text>
               Not all variables possess a time axis
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR No z axis for variable</key>
              <text>
                Not all variables possess a Z axis
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR No x axis for variable</key>
              <text>
                Not all variables possess a Longitude axis
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR No y axis for variable</key>
              <text>
                Not all variables possess a Latitude axis
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR The t region is too small. It reduces to a single point.</key>
              <text>
                The time region requested reduces to a single point and is too small.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR The z region is too small. It reduces to a single point.</key>
              <text>
                The Z region requested reduces to a single point and is too small.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR The x region is too small. It reduces to a single point.</key>
              <text>
                The longitude region requested reduces to a single point and is too small.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR The y region is too small. It reduces to a single point.</key>
              <text>
                The latitude region requested reduces to a single point and is too small.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**ERROR</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**Error</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**TMAP ERR</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>STOP</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>Segmentation</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>No such</key>
              <text>
                An error occurred in the service that was creating your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**Internet data error</key>
              <text>
                A remote server was unable to deliver the data LAS needs to make your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**netCDF error</key>
              <text>
                A remote server was unable to deliver the data LAS needs to make your product.
              </text>
           </message>
           <message type="startsWith" level="error">
              <key>**Internet Data error</key>
              <text>
                A remote server was unable to deliver the data LAS needs to make your product.
              </text>
           </message>
    </messages>
    <environment>
        <variable>
            <name>FER_DIR</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72</value>
        </variable>
        <variable>
            <name>FER_DESCR</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72/fer_dsets/descr</value>
             <value>des</value>
        </variable>
        <variable>
            <name>FER_DATA</name>
             <value>.</value>
             <value>./data</value>
             <value>/opt/atoll/products/ferret_6.72/examples</value>
             <value>/opt/atoll/products/ferret_6.72/fer_dsets/data</value>
             <value>/opt/atoll/products/ferret_6.72/go</value>
        </variable>
        <variable>
            <name>FER_GRIDS</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72/fer_dsets/grids</value>
        </variable>
        <variable>
            <name>FER_PALETTE</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72/ppl</value>
             <value>jnls</value>
             <value>jnls/insitu</value>
             <value>jnls/section</value>
             <value>scripts</value>
        </variable>
        <variable>
            <name>FER_GO</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72/contrib</value>
             <value>/opt/atoll/products/ferret_6.72/examples</value>
             <value>/opt/atoll/products/ferret_6.72/go</value>
             <value>jnls</value>
             <value>jnls/insitu</value>
             <value>jnls/section</value>
             <value>scripts</value>
        </variable>
        <variable>
            <name>PLOTFONTS</name>
             <value>/opt/atoll/products/ferret_6.72/ppl/fonts</value>
        </variable>
        <variable>
            <name>FER_EXTERNAL_FUNCTIONS</name>
             <value>.</value>
             <value>/opt/atoll/products/ferret_6.72/ext_func/libs</value>
        </variable>
        <variable>
            <name>DODS_CONF</name>
             <value>/opt/atoll/tomcat-las-aviso/content/las/conf/server/dods/.dodsrc</value>
        </variable>
    </environment>
</application>
-----------------------------------------------------------------------------------------------

Le 24/08/2012 20:42, Ansley Manke a écrit :
Hi Sebastien,
I think that the server at Aviso must be installed on a system where for some reason they are not able to access the remote dataset http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc

I have written to the contact email address for the server at Aviso asking whether that is the case. The solution would be to provide a patch to the Ferret scripts that LAS runs, which will check whether the dataset can be opened, and if not revert to a somewhat lower resolution bathymetry/topography dataset which will be local on their system as it's part of the Ferret distribution and will have been installed when they installed Ferret and LAS.

Ansley

On 8/24/2012 3:06 AM, Sebastien Lebosse wrote:

What is even more strange is that I try to run the corresponding LAS Ferret script from the command line as specified here http://ferret.pmel.noaa.gov/LAS/documentation/installer-documentation/errors-and-debugging/running-an-las-ferret-script-from-the-command-line/, there is no problem and I can generate my map on the black sea area.

regards,
Sébastien

Le 24/08/2012 11:11, Sebastien Lebosse a écrit :
Hi Ansley,

Many thanks for your timely response. Indeed there is no problem with land outline option but for many reasons I need to resolve this problem (if it's possible !). As far as I known, there is no restricts access to external Thredds Data Servers.

I noticed that this problem appears after a certain level of zoom. If I select "DT - Delayed time Data / DT - Global / DT - Global - (Maps of) Sea Level Anomalies And Formal Mapping Error" in http://las.aviso.oceanobs.com/las/getUI.do, ferret systematically generates the maps with the global region but when I zoom to the black sea area the maps generation become impossible.

In attached file you can see the las.log extract for these both requests. I don't know why (in this extract) when I plot a little region (like the black sea) the LAS systematically repeats the line "ProductServerAction - START: http://atoll-las.vlandata.cls.fr:51080/las/ProductSe....." and after some time fails.

At the moment, I don't understand what is different between this both request and I don't understand why we don't have the same behavior with your LAS http://oos.soest.hawaii.edu/las/ .

If you have an idea or an advice to help address this issue, I thank you in advance.

regards,
Sébastien

Le 22/08/2012 19:26, Ansley Manke a écrit :
Hi Sebastien,
I can use that dataset from here, in our LAS servers, and it's working fine for small-region plots using filled land at other LAS installations, such as http://oos.soest.hawaii.edu/las/, so it's a bit hard to diagnose. I wonder if Aviso has something in place that restricts access to thredds servers outside their own.

As you may know, a workaround is to select land outline in the plot options; and in fact some of the datsets on that server at aviso are configured that way. For instance, I went to this selection,
Select by geographic area / Global ocean / Delayed Time - Global / Delayed Time - Global - (Maps of) Absolute Geostrophic Velocities, and this data makes a land outline.

-Ansley

On 8/22/2012 7:44 AM, Sebastien Lebosse wrote:

Dear LAS team,

http://las.aviso.oceanobs.com/las/getUI.do uses the las.v7.3.

This LAS cannot generate maps with option land fill style = Filled because the essential http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc resource for ferret scripts is not available or impossible to use.

What should I do to troubleshoot the problem ?

below an extract of the las.log :
----------------------------------------------
(...)
! If a tiny region then use smith_sandwell topo from PMEL server
IF `fland_area LT 0.005` THEN
!-> IF 1 THEN

   SET DATA/SAVE

   DEF SYM xlo_hack ($xaxis_min%0%)
!-> DEF SYM xlo_hack 27.000000
   DEF SYM xhi_hack ($xaxis_max%360%)
!-> DEF SYM xhi_hack 42.00000
   DEF SYM ylo_hack ($yaxis_min%-90%)
!-> DEF SYM ylo_hack 40.000000
   DEF SYM yhi_hack ($yaxis_max%90%)
!-> DEF SYM yhi_hack 47.00000

   DEF SYM map_reg = [x=($xlo_hack):($xhi_hack),y=($ylo_hack):($yhi_hack)]
!-> DEF SYM map_reg = [x=27.000000:42.00000,y=40.000000:47.00000]

   USE "http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc"
LAS 2012-08-22T14:16:21.006 +0000 INFO - ProductServerAction - START: http://atoll-las.vlandata.cls.fr:51080/las/ProductServer.do?xml=<?xml version="1.0"?><lasRequest href="" moz-do-not-send="true">"file:las.xml"><link match="/lasdata/operations/operation[@ID='Plot_2D_XY_zoom']"></link><properties><ferret><view>xy</view><land_type>filled</land_type><set_aspect>default</set_aspect><mark_grid>no</mark_grid><contour_levels></contour_levels><fill_levels></fill_levels><contour_style>default</contour_style><palette>default</palette><deg_min_sec>default</deg_min_sec><margins>default</margins><use_graticules>default</use_graticules><size>0.5</size><image_format>default</image_format><interpolate_data>false</interpolate_data><_expression_></_expression_></ferret></properties><args><link match="/lasdata/datasets/dataset-duacs-dt-upd-blacksea-merged-msla-h-latlon-switched/variables/grid_0001"></link><region><range type="y" low="40" high="47"></range><range type="x" low="27" high="42"></range><point type="t" v="08-FEB-2012 00:00:00"></point></region></args></lasRequest>
LAS 2012-08-22T14:16:21.075 +0000 INFO - FerretBackendService - START: Plot_2D_XY
LAS 2012-08-22T14:16:45.453 +0000 INFO - ProductServerAction - START: http://atoll-las.vlandata.cls.fr:51080/las/ProductServer.do?xml=<?xml version="1.0"?><lasRequest href="" moz-do-not-send="true">"file:las.xml"><link match="/lasdata/operations/operation[@ID='Plot_2D_XY_zoom']"></link><properties><ferret><view>xy</view><land_type>filled</land_type><set_aspect>default</set_aspect><mark_grid>no</mark_grid><contour_levels></contour_levels><fill_levels></fill_levels><contour_style>default</contour_style><palette>default</palette><deg_min_sec>default</deg_min_sec><margins>default</margins><use_graticules>default</use_graticules><size>0.5</size><image_format>default</image_format><interpolate_data>false</interpolate_data><_expression_></_expression_></ferret></properties><args><link match="/lasdata/datasets/dataset-duacs-dt-upd-blacksea-merged-msla-h-latlon-switched/variables/grid_0001"></link><region><range type="y" low="40" high="47"></range><range type="x" low="27" high="42"></range><point type="t" v="08-FEB-2012 00:00:00"></point></region></args></lasRequest>
(...)
LAS 2012-08-22T14:19:30.876 +0000 ERROR - FerretBackendService - Ferret backend request failed: A remote server was unable to deliver the data LAS needs to make your product.
LAS 2012-08-22T14:19:30.877 +0000 INFO - FerretBackendService - END:   Plot_2D_XY
LAS 2012-08-22T14:19:33.229 +0000 INFO - ProductServerAction - START: http://atoll-las.vlandata.cls.fr:51080/las/ProductServer.do?xml=<?xml version="1.0"?><lasRequest href="" moz-do-not-send="true">"file:las.xml"><link match="/lasdata/operations/operation[@ID='Plot_2D_XY_zoom']"></link><properties><ferret><view>xy</view><land_type>filled</land_type><set_aspect>default</set_aspect><mark_grid>no</mark_grid><contour_levels></contour_levels><fill_levels></fill_levels><contour_style>default</contour_style><palette>default</palette><deg_min_sec>default</deg_min_sec><margins>default</margins><use_graticules>default</use_graticules><size>0.5</size><image_format>default</image_format><interpolate_data>false</interpolate_data><_expression_></_expression_></ferret></properties><args><link match="/lasdata/datasets/dataset-duacs-dt-upd-blacksea-merged-msla-h-latlon-switched/variables/grid_0001"></link><region><range type="y" low="40" high="47"></range><range type="x" low="27" high="42"></range><point type="t" v="08-FEB-2012 00:00:00"></point></region></args></lasRequest>
LAS 2012-08-22T14:19:33.249 +0000 ERROR - LASAction - A remote server was unable to deliver the data LAS needs to make your product.
LAS 2012-08-22T14:19:33.249 +0000 ERROR - LASAction - **Internet Data error
             NetCDF: I/O failure (OPeNDAP/netCDF Error code -68)
             Data set: http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc
SET DAT/FORM=CDF "http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/smith_sandwell_topo_v8_2.nc"
Command file, command group, or REPEAT execution aborted
ERROR RUNNING SCRIPT
----------------------------------------------

Thanks in advance,
Sébastien

Cliquez ici si ce message est indésirable (pourriel).

Re: [las_users] problem with smith_sandwell_topo_v8_2.nc resource