HI Paul,you can use repeat command with the set memory command so it will clear the ram at every time step.suppose i am having 4000 time steps of variable test_var and i want to save them. I will use this repeat command like thisfor your problem you can use it.
repeat/l=2:7305 (\
save/append/file = "/archive/Paul.Goddard/CM2.6/Storm_Surge_Project/CM2.6_Contr ol_SSH_DETRENDED_01810101- " SSH_DETRENDED[l=`l`]; set mem/size=2000)02001231.nc regards, saurabh--On Fri, Apr 21, 2017 at 11:57 AM, Paul Goddard <pgoddard@xxxxxxxxxxxxxxxxx> wrote:Any ideas on the best way to save such large data? Given that this is the resolution of many of the ocean models for CMIP6, I think I better learn a good way to complete these tasks.In the past, I was able to save large data by repeating (looping) over the time dimension. However, since a detrending calculation must happen as well; it is taking too long to even save the first year (going on 2 hours... and may crash before it saves l=1..)Hello,I am attempting to detrend SSH data over time at each ocean grid cell. The problem is that the data are very large, the grid is i=1:3600 , j=1:2700, and l=1:7305.
Tthank you in advance, Paul
Here is my script:
can data/all
can var/all
can win/all
use "./CM2.6_Control_SSH_01810101-02001231.nc "
set memory/size=99999
let P = T[GT=SSH,l=1:7305]
let Q = SSH[i=1:3600,j=1:2700,l=1:7305]
SET GRID Q
GO regresst
let SSH_Detrended = SSH[i=1:3600,j=1:2700,l=1:7305] - qhat[i=1:3600,j=1:2700,l=1:730 5]
!Control save
save/clobber/file = "/archive/Paul.Goddard/CM2.6/Storm_Surge_Project/CM2.6_Contr ol_SSH_DETRENDED_01810101- " SSH_DETRENDED[l=1]02001231.nc
repeat/l=2:7305 (\
save/append/file = "/archive/Paul.Goddard/CM2.6/Storm_Surge_Project/CM2.6_Contr ol_SSH_DETRENDED_01810101- " SSH_DETRENDED[l=`l`])02001231.nc REGARDSSaurabh RathoreResearch Scholar (PhD.)Centre For Oceans, Rivers, Atmosphere & Land Science TechnologyIndian Institute Of Technology, Kharagpurcontact :- 91- 8345984434