Re: [ferret_users] Wrong result in division

To: Ferret <ferret_users@xxxxxxxx>

Subject: Re: [ferret_users] Wrong result in division

From: "Fiedler, Russell (O&A, Hobart)" <Russell.Fiedler@xxxxxxxx>

Date: Tue, 21 Jul 2020 02:02:16 +0000

Accept-language: en-AU, en-US

Arc-authentication-results: i=5; mx.google.com; dkim=pass header.i=@csiro.au header.s=dkim header.b=RK0K+YuB; dkim=pass header.i=@CSIROAU.onmicrosoft.com header.s=selector1-CSIROAU-onmicrosoft-com header.b=RRF0b+sK; arc=pass (i=3 dkim=pass dkdomain=csiro.au dkim=pass dkdomain=csiroau.onmicrosoft.com); spf=fail (google.com: domain of prvs=46447de7b=russell.fiedler@xxxxxxxx does not designate 209.85.222.199 as permitted sender) smtp.mailfrom="prvs=46447de7b=russell.fiedler@xxxxxxxx"

Arc-authentication-results: i=4; mx.google.com; dkim=pass header.i=@csiro.au header.s=dkim header.b=RK0K+YuB; dkim=pass header.i=@CSIROAU.onmicrosoft.com header.s=selector1-CSIROAU-onmicrosoft-com header.b=RRF0b+sK; arc=pass (i=3 dkim=pass dkdomain=csiro.au dkim=pass dkdomain=csiroau.onmicrosoft.com); spf=fail (google.com: domain of prvs=46447de7b=russell.fiedler@xxxxxxxx does not designate 209.85.222.199 as permitted sender) smtp.mailfrom="prvs=46447de7b=russell.fiedler@xxxxxxxx"

Arc-authentication-results: i=3; mx.google.com; dkim=pass header.i=@csiro.au header.s=dkim header.b=RK0K+YuB; dkim=pass header.i=@CSIROAU.onmicrosoft.com header.s=selector1-CSIROAU-onmicrosoft-com header.b=RRF0b+sK; arc=pass (i=2 spf=pass spfdomain=csiro.au dkim=pass dkdomain=csiro.au dkim=pass dkdomain=csiroau.onmicrosoft.com dmarc=pass fromdomain=csiro.au); spf=fail (google.com: domain of prvs=46447de7b=russell.fiedler@xxxxxxxx does not designate 65.153.203.169 as permitted sender) smtp.mailfrom="prvs=46447de7b=Russell.Fiedler@xxxxxxxx"

Arc-authentication-results: i=2; mx.google.com; dkim=pass header.i=@csiro.au header.s=dkim header.b=RK0K+YuB; dkim=pass header.i=@CSIROAU.onmicrosoft.com header.s=selector1-CSIROAU-onmicrosoft-com header.b=RRF0b+sK; arc=pass (i=1 spf=pass spfdomain=csiro.au dkim=pass dkdomain=csiro.au dmarc=pass fromdomain=csiro.au); spf=pass (google.com: domain of prvs=46447de7b=russell.fiedler@xxxxxxxx designates 2405:b000:e00:257::7:43 as permitted sender) smtp.mailfrom="prvs=46447de7b=Russell.Fiedler@xxxxxxxx"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=csiro.au

Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=csiro.au; dmarc=pass action=none header.from=csiro.au; dkim=pass header.d=csiro.au; arc=none

Arc-message-signature: i=5; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-archive:list-help:list-post:list-id:mailing-list:precedence :message-id:mime-version:content-language:accept-language :in-reply-to:references:date:thread-index:thread-topic:subject:to :from:ironport-phdr:ironport-sdr:sender:dkim-signature; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=ELR9G7MueMTKCs9vBkirgwsmUD2pr6F1UUOzIwX1X+rfAMcfaNy2Lk5LI4TAzNXfUV goPGzONkhIDUCxifzGVt7StoZnGYTkPlYZzzV5mtxbJcNQUwqxC/pA309kBwHZBpG5Jo f60hqmmIfGq7ikL+8H9+pRsK/yoDFsysPFB8hnjV/rQ5cwdyTMwot55fNtQSMyAyd2Da 04+euP+xOPBRO2mEssptnLW8VNrAi78pXe6Fg9/hxxkkCN/KQrEAJnMgGg5yyN95LQCs LoTRBeOMAraSH3nfa7/sZnKshJXBVBygWyDq66WtwdDXLbHNSfNBfIHxirrO2s/QVabx geOA==

Arc-message-signature: i=4; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:mime-version:content-language:accept-language :in-reply-to:references:date:thread-index:thread-topic:subject:to :from:dkim-signature:ironport-phdr:ironport-sdr:dkim-signature; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=y4hnFinhRM781e4KSfSsALOnlNuFzLolBR+dqUrCN5PIgIwR/HOF/26hz61KW7HZfR nRh4EndfLUI23V/vPqVWt9XsN/LvqX3MPdvNIiAa+RWTVljyMdm1NP8ghepO9YtERJRB 7uzmwFA+POEP53b/OB7rOgDZ1K3KlE/kbHI/qrU+UC52MJM5wYsYzSD+dRaDUem/I1Ko vqP4n/URZORpFKbINwvOy0Hcu23A5ywwr93pK5dZdVAtv9m20/EG4fGAuNNMcthVcjPO TaUiMzpSXxdDCFhAsFqEir7Xm/Su8f4s8Gwqgzi+p0XxAn7MiW6sKUJmlNnsam8/TeKh Edmg==

Arc-message-signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-language:accept-language:in-reply-to :references:message-id:date:thread-index:thread-topic:subject:to :from:dkim-signature:ironport-phdr:ironport-sdr:dkim-signature; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=k3LN/Ka3jJmHvVJsLbtgh1c2cGj2xQWPCNuCaoVVPDeajh4xoqMXCP4tArBKwZiApC 18G7We38n/lbDdp5KJzgC2V5Wcpeb9QiFFdmS2O6uLgp3fsydWRZbc+DiH7vD+ujTNsb X2fUeGwGzJA8fsGQSdc+Pv33dzpsm7FEMyNUO6NhzZmmgZns6Npn7h1JhAwBl024aCtJ 0dF+qIXzqnBms6QpPPz+78469MipFXGzZ2xDoBjFteIuJExQCYSKbyyEuy4gMsuqRCbU gnIge+uIrLEY7B6ir+knWaG4uWVsq1ZfaE2BUwqyeQTSIIttM7R38zeyBhlEZhcc00ht Uk2Q==

Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-language:accept-language:in-reply-to :references:message-id:date:thread-index:thread-topic:subject:to :from:dkim-signature:ironport-phdr:ironport-sdr:dkim-signature; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=tKFcpmuWgHAtI//tbJGYCrEyihf1y8DOhMQ1WrZXJBtTwiLdIJ2nW6lWhm/L0ssrl6 YdXhC1FySaKd2ofpVQ5SXRVbwId1lHSSevOsSdQNLTw1Nori4b8r098YnceO/kUUs1NP d3mCQFwtYXPJHriLOo+xXGQrPzpClEJXcdf946hgF06VYgM/i8UKhJ6A1Bu/tAs0O4+E WJug9pn9iiKPk0bk2DOoU9O/4F3pTcmHllkkw1wyiyQw0Wbn5A9EUhNYARLA8Tuqz+if d0rIChVOyB7VV/X12ROGxrIcbabz89f1WkWlZ0QB1LYv1t/8Xqlib4sVuiwybOzcKR5O GRkg==

Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=gzfdpoaJqa0zgujSAwpjDJgE347GR2lPz3n43DUJk5fNnWCZezAjSDJYA0DcGarNhwVpuMfeVeFocAzAqjCqJZDlLZAk/kyMus8cPanE6zWLq9wvZtscxywRHhBMA0NhbUQqS7Sp77DxPMFl3NdpAP8uTvHBhxs2B9UbyeoH/Sk5j3r/+KjdefQVRjjkFr52fGivncm5TaEYCQYVuCziaqZCb+W2+SCqsHa4lMCcrzbGH3qXNs5l8mv2UD8dw7TTlPFJtWLkVRkfEKPAqsVkbfwtrCkswEY5exqjoxbt2bD0DzC/EnYzLmEgPBX9UQ6CNxT8my/f6ihXQKvfwd2hAQ==

Arc-seal: i=5; a=rsa-sha256; t=1595296979; cv=pass; d=google.com; s=arc-20160816; b=t/IZu4Tn3Gv5Vu6/BYMIDaqRkvriJVuaujz65HQbXm4vxekqbfaZ0tq642ZO3vu/XH KLjIKJ2Qe12rzkCf45Mo6BbKemQba0QxOsS3aNroYN4iioTluWxlLSHhxNrKP8KnPJD2 unOCVXv5PbKudBYHNchiBT5kxFEjfsKMxE/qqPgThzD+VcPQS1j1w8whYEfFnuwip/4H POeI62aPilRrypHEh4XKKGfvcCgf+hrryRtgwWufLahOr/Yw0jPAHMMSHLAhIQGdyFV0 dX2I0FIexqOsXo0IUMgeUrq1tIp5CwQyqvWG3b4lFSw7wJ2i3/W4j77r+TpvtYjCzGoi YtAA==

Arc-seal: i=4; a=rsa-sha256; t=1595296979; cv=pass; d=google.com; s=arc-20160816; b=Li/PlXWWtND5rvl2nxHp2ny0+V6vnBO4CemvpfnFqfJ9wlprfpJHplyM7n+cMD24q3 kmPUFbFQmNnZ5GI6hJEYaBmiwsmEKzMEWQEsdxrEzToXQHTILj0m+n5eDKUnFw5W7xMU ZHTm60EwtyE44eCOBiwO/kXkyUlvJALHxxhlCZy8+/9toeCPpxSD62jN9qWq/7hyn3jT j/6p41qquDANhH1UqAoaUztAYzQMrxg9Gj1w6NbhqeEhkEUvtbMwMw3X/UPVGO706feQ SGbrAQ3o1dWG18u702mjtUMfsnMIEzSTCRR+w8gl2DllzzxdXW7+Hz3x0xXA7CT2+IkE aIQw==

Arc-seal: i=3; a=rsa-sha256; t=1595296976; cv=pass; d=google.com; s=arc-20160816; b=VAFItezVi6YmQxggiCaWs0WGaz3U3nIQMkspAzrCFYMvf5it7OLhr4btV3Vg0AQJ9c hdqdFTsGxeufalb5D2xsaexdaJbBrQT5iBUy2eGNV5apc284C5Z7jNEMP+i/oiN9fC58 nZ2dq5/ZjoIDZoeYbPPzi7cMbXuFCSc1pQYvKLzzdSRiMMACDMdNkNnG7OMuR6rizYb1 DJ2DwixfyV3/ZieRmovvbiIOwPD4lk7zHLBUj277/ugsKeRRtELwj5IzAvkyvscwjopS UAI6Et8DA04LQlPqb58TOhE3PU8/WaLIJLH7cuPxTt0FHXrdqZYCFhcTWDE9V7h1LLrr UvoQ==

Arc-seal: i=2; a=rsa-sha256; t=1595296973; cv=pass; d=google.com; s=arc-20160816; b=vW3WseDWWkJgUl7doOR9AFjK8ae0/UfSb3WhTiUVX/I16G3IZc0Y1FkSmPi291TGQt +tIHlJkaSgJKp1cxnqCCEw4iBSPyaYqdcBYjx0LEjNV/aIVtZuB/BH0MI/wMCUi8bHtz jsVIaAdUAGtiG1IaKtft47/FKCD/ltQqOMMaTsgI3xvjtJBUg72PhCTh7KDEPkRl7z9X pLaQ0X0cR4o1WqT/pxgQi/q0k3gQUXRdmK2nZrBh16CCqn6gMwmWvFwtefOkUZ7vT2IU fZLUwM0D4Q1fOGljVfSFkMR/kZy+RhML15kWvs73UEwokFx71vQPve6UnuxIJNnIB7rM Apww==

Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iwLuZQWlOdFBq+Fq+2p2VnAgscFB3GFc8w7EVnyVTZUoIYd7AV7DT8quj0sT67wxpplMe+oACMTqidKnNqkPZ0DiBZw8ZhDouqLMv9asbIjfq4lBBmRxxbhORENX43w2Ur0xUMbRDUAiXxGk0j4Ify07IJUYbPtWZQJ0JQTmdHBWxrOoI1QtK8ddgAZc4emE8srAkaO9Af4Z4BJnlxIPwGoU0fFiqS3oXa8oNFCT7c6bM0ABIameT4Ob9nvnLIZ4wLDpY98BAJUS5ZngdHXvUJaH+CROh47o+GKFuoKyTgKZzllIn+Lawx8+ZhlNJwFze30KSo0cfCXYrOwMYWyIew==

Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=noaa.gov; s=google; h=sender:ironport-sdr:ironport-phdr:from:to:subject:thread-topic :thread-index:date:references:in-reply-to:accept-language :content-language:mime-version:message-id:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive; bh=FSgIRqQ/EZbWlPLxYtjgvrsYLfyWOUnlMxpj3DZU71o=; b=RFG9lDUCAKGg49b9qmKEyR7wvePo574H73LilwePTPdVCPwl6wqR8+KqyUsRHqQz+n 9zepehqMrBRMfVmK5SxWbSGt2wSQhb/2eaWlrcPqX6sNctgPWVt9MpjPuocPNE6gvyum 9nb7esxr6nJR5eo25tOqVPWYlIYZxufqZRvI4KeFebOFPhHtTMnl7o4sibx1tiDytXq8 rDkiYlK0kd2umPb3twe8+0PiFsLTSmlrzBdpO6tn6Ea1bRhV79uAPxwpeSpQ1fPFZeB1 iVC7/ivVIqFwVsYkva1+Py0RVpryvm23pBU2ysgRRY+iBVXWIYWA5NxEwoJjA9Vzam3G L4sw==

In-reply-to: <CAEwQpaL5Zc-JtpoFLQr8q1Lb3w+PoWAtbW-rZtOxhNwquRbM+w@mail.gmail.com>

Ironport-phdr: 9a23:Ti909hzH1JyXRq3XCy+N+z0EezQntrPoPwUc9psgjfdUf7+++4j5ZRWGt/NqhVaPUoLFuLpIiOvT5qbnX2FIoZOMq2sLf5EEURgZwd4XkAotDI/gawX7IffmYjZ8EJFEU1lorHe8OEETF8HgNBXep3So5msUHRPyfQN+OuXyHNvUiMK6n+C/8pHeeUNGnj24NLhpLQ+x6wzM5cQ=

Ironport-sdr: QSryOdkEeGN4Nho+C8fn2myG437pbuxp/rarik6Bl12yjf0iyCmX9Wtc90Kv+lj4ugdRKiVZL2 3NHxBzIE1eBA==

List-archive: <https://groups.google.com/a/noaa.gov/group/ferret_users/>

List-help: <https://support.google.com/a/noaa.gov/bin/topic.py?topic=25838>, <mailto:ferret_users+help@noaa.gov>

List-id: <ferret_users.noaa.gov>

List-post: <https://groups.google.com/a/noaa.gov/group/ferret_users/post>, <mailto:ferret_users@noaa.gov>

Mailing-list: list ferret_users@xxxxxxxx; contact ferret_users+owners@xxxxxxxx

References: <0fde9cf2-7047-98ca-0780-1fc3a2fe9f75.ref@yahoo.it> <9f0d2db9-7a55-dc47-da5a-6f88bc184976@yahoo.it>,<CAEwQpaL5Zc-JtpoFLQr8q1Lb3w+PoWAtbW-rZtOxhNwquRbM+w@mail.gmail.com>

Sender: owner-ferret_users@xxxxxxxx

Thread-index: AQHWXUlrVE2MEctwxUiQXMoNXBvoU6kPzsEAgAF7AeQ=

Thread-topic: [ferret_users] Wrong result in division

Ryo,

I believe that Fabio is referring to the results for j=31 and j=35 at 35E. 0.049/0.039 is definitely not 5.54!

My guess is that there may be some masking or regridding issues but really more info is needed to diagnose this.

Russ

From: owner-ferret_users@xxxxxxxx <owner-ferret_users@xxxxxxxx> on behalf of Ryo Furue <furue@xxxxxxxxxx>
Sent: Monday, 20 July 2020 1:20 PM
To: Fabio <xxtraloud@xxxxxxxx>
Cc: ferret users <ferret_users@xxxxxxxx>
Subject: Re: [ferret_users] Wrong result in division

Fabio,

On Sun, Jul 19, 2020 at 6:21 AM 'Fabio' via _OAR PMEL Ferret Users <ferret_users@xxxxxxxx> wrote:

let result = myvar1/myvar2

MYVAR1 MYVAR2 RESULT
---- J:30
25E / 103: 0.0054 0.0049 1.09

I guess you mean that 0.0054/0.0049 = 1.102 , not 1.09 .

I guess that it's not that the calculation is incorrect but that the values listed on the screen are misleading. Below I reproduce your result:

yes? let myvar1 = 0.00536
yes? let myvar2 = 0.0049
yes? let result = myvar1/myvar2
yes? list/format="(2(1X,F6.4),1X,F4.2)" myvar1, myvar2, result
Column 1: MYVAR1 is 0.00536
Column 2: MYVAR2 is 0.0049
Column 3: RESULT is MYVAR1/MYVAR2
0.0054 0.0049 1.09

Note how the value 0.00536 is printed as 0.0054 .

So, you want to see what happens if you increase the numbers of digits when you print the values on your screen.

Ryo