Error and assertion when viewing EAM (E3SM) initial condition file

[whannah1]

I ran into a couple problems trying visualize some initial condition files in the E3SM inputdata repo. The files can be quite different, but a few of them have been particularly problematic, including this one => atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc

When I first tried to open it with ncvis I got this error about the swband variable:

> ncvis /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc
NcVis 2022.10.26 Paul A. Ullrich
ERROR: NetCDF fileset contains a dimension variable "swband" that is non-monotone

After removing that with a simple nco command:

ncks -x -v swband /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc eam_i_rcemip_ne45np4_L60_c20210823_tmp.nc

I tried again and got this error:

...
..97% complete
..98% complete
..99% complete
..Done
Initializing the GridDataSampler took 31ms
EXCEPTION (/global/homes/p/paullric/ncvis/src/wxImagePanel.cpp, Line 648) Assertion failure

I'm trying to track down some issues with these files used for RCE experiments, so it's possible that there is a NaN value in the data, but either way it would be nice to have an option to ignore certain variables like swband.

[paullric]

I switched the ERROR to a WARNING on Cori and recompiled. That got us past the first issue. As for the assertion failure, it seems that there are two unstructured grid dimensions in the file: ncol and ncol_d. Right now ncvis is detecting ncol_d as the unstructured grid dimension used for sampling. Would it be possible to separate the variables using ncol from the variables using ncol_d? i.e., ncks -v lat,lon,area,ICEFRAC,SICTHK,SNOWHICE,TS1,TS2,TS3,TS4,TSICE /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc temp.nc I can put it on the “to do” list to support files with multiple unstructured dimensions.

…

On Dec 13, 2022, at 8:48 AM, Walter Hannah ***@***.***> wrote: I ran into a couple problems trying visualize some initial condition files in the E3SM inputdata repo. The files can be quite different, but a few of them have been particularly problematic, including this one => atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc When I first tried to open it with ncvis I got this error about the swband variable: > ncvis /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc NcVis 2022.10.26 Paul A. Ullrich ERROR: NetCDF fileset contains a dimension variable "swband" that is non-monotone After removing that with a simple nco command: ncks -x -v swband /global/cfs/cdirs/e3sm/inputdata/atm/cam/inic/homme/eam_i_rcemip_ne45np4_L60_c20210823.nc eam_i_rcemip_ne45np4_L60_c20210823_tmp.nc I tried again and got this error: ... ..97% complete ..98% complete ..99% complete ..Done Initializing the GridDataSampler took 31ms EXCEPTION (/global/homes/p/paullric/ncvis/src/wxImagePanel.cpp, Line 648) Assertion failure I'm trying to track down some issues with these files used for RCE experiments, so it's possible that there is a NaN value in the data, but either way it would be nice to have an option to ignore certain variables like swband. — Reply to this email directly, view it on GitHub <#11>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIVPZETJO6UM4SQMPJP3S3WNCSHTANCNFSM6AAAAAAS5NZOB4>. You are receiving this because you are subscribed to this thread.

[whannah1]

Thanks Paul.

As for the assertion, I'm fine working around that problem with a temporary file, but can you put a warning in to explain what's going on? I'm thinking other people will likely hit this in the future, so it would be to have an informative error message.

[zarzycki]

This is probably obvious, but FYI this is a challenge with initial conditions for SE/HOMME using physgrid because some variables are initialized on the physgrid and others on the GLL nodes. IIRC, all history output in E3SM and CESM using SE/HOMME is dumped on the physgrid so you'll only have one ncol there.

[whannah1]

@zarzycki I'm actually not sure that we "need" the physgrid data in the atmospheric IC file. In the files I'm looking at the only physgrid data is for surface data, like "TSICE", which I don't think we actually use... not sure though, maybe it's used for the first call to radiation?

On the other hand, we do need both grids in the topography file, so that might be a better case of having the ability to handle multiple grids in the same file to do quick sanity checks after generating new topo files. But that's also somewhat of an edge case.

Having a way to set the coordinate of interest (i.e. "ncol_d") and ignore any data that doesn't use that dimension would be a nice feature to have.

[zarzycki]

That's probably a good short-term solution (having a user option to specify the unstructured dimension at the command line or via the GUI). ncremap does something similar (i.e., it has a default "list" of unstructured dim names but you can also specify which one corresponds.

The MPAS dual grid would be another situation where there are multiple unstructured dimensions on a single file (technically you can have the model dump winds at the cell edges and scalars at the cell center). Although they also have an output stream that is the edge data mapped to cell center which is what most folks I know use for diagnostics.

[rljacob]

Curious why ncol_d was chosen instead of ncol when both were present.

[whannah1]

@rljacob the variable I was trying to plot (PS) has the ncol_d dimension.

[mt5555]

similar issue when viewing EAMxx output for a file that contains both "pg2" and "np4" data. In this case, the coordinate dimension variables are renamed lat_pg2, lon_pg2, lat_gll, lon_gll.

One can use "ncrename" to rename one pair to lat/lon, and then the associated variables can be visualized.

The CAM (and EAM) convention in this case is that the physics grid coordinates are always lat,lon, while the dynamics grid, if it is different than the physics grid is lat_d,lon_d.

Suggestion: can ncvis handle all these cases by adding the _d, _pg2 and _gll options here in the code:

ncvis/src/wxNcVisFrame.cpp

Lines 396 to 406 in 0a9d36a

	std::vector<std::string> vecCommonLonVarNames;
	vecCommonLonVarNames.push_back("lon");
	vecCommonLonVarNames.push_back("longitude");
	vecCommonLonVarNames.push_back("lonCell");
	vecCommonLonVarNames.push_back("mesh_node_x");
	std::vector<std::string> vecCommonLatVarNames;
	vecCommonLatVarNames.push_back("lat");
	vecCommonLatVarNames.push_back("latitude");
	vecCommonLatVarNames.push_back("latCell");
	vecCommonLatVarNames.push_back("mesh_node_y");

[paullric]

ncvis also looks for standard_name == “longitude”. Are there any attributes for those dimension variables that could be used to identify that they’re latitude/longitude?

…

On May 27, 2025, at 7:16 AM, Mark Taylor ***@***.***> wrote: mt5555 left a comment (SEATStandards/ncvis#11) <#11 (comment)> similar issue when viewing EAMxx output for a file that contains both "pg2" and "np4" data. In this case, the coordinate dimension variables are renamed lat_pg2, lon_pg2, lat_gll, lon_gll. One can use "ncrename" to rename one pair to lat/lon, and then the associated variables can be visualized. The CAM (and EAM) convention in this case is that the physics grid coordinates are always lat,lon, while the dynamics grid, if it is different than the physics grid is lat_d,lon_d. Suggestion: can ncvis handle all these cases by adding the _d, _pg2 and _gll options here in the code: https://github.com/SEATStandards/ncvis/blob/0a9d36a6ad3cc02c1fc7371ba935db32d9519664/src/wxNcVisFrame.cpp#L396-L406 — Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIVPZDVYQCI7OUMXHS5KYL3ARXTPAVCNFSM6AAAAAB6AGBH4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJSGY4DENJTGU>. You are receiving this because you commented.

[mt5555]

	float lon_gll(ncol_d) ;
		lon_gll:units = "rad" ;
		lon_gll:_FillValue = 3.402824e+33f ;
		lon_gll:long_name = "MISSING" ;
		lon_gll:standard_name = "MISSING" ;

So we just change standard_name to longitude?

Can both lon_gll and lon_pg2 have the same standard_name? (and then will ncvis find the one with the correct dimension?)

edit: I'll check this myself, since i think I can add those attributes directly to the file

[rljacob]

Yes fixing the standard name should be enough. Coupler history has multiple lat/lon coordinates and ncvis works on those variables.

[mt5555]

Added the standard_name attribute (see below).

It works for the variables on the the gll grid, but not for the variables on the pg2 grid.

        float lat_gll(ncol_d) ;
                lat_gll:units = "rad" ;
                lat_gll:_FillValue = 3.402824e+33f ;
                lat_gll:long_name = "MISSING" ;
                lat_gll:standard_name = "latitude" ;
        float lon_gll(ncol_d) ;
                lon_gll:units = "rad" ;
                lon_gll:_FillValue = 3.402824e+33f ;
                lon_gll:long_name = "MISSING" ;
                lon_gll:standard_name = "longitude" ;
        float lat_pg2(ncol) ;
                lat_pg2:units = "rad" ;
                lat_pg2:_FillValue = 3.402824e+33f ;
                lat_pg2:long_name = "MISSING" ;
                lat_pg2:standard_name = "latitude" ;
        float lon_pg2(ncol) ;
                lon_pg2:units = "rad" ;
                lon_pg2:_FillValue = 3.402824e+33f ;
                lon_pg2:long_name = "MISSING" ;
                lon_pg2:standard_name = "longitude" ;

[mt5555]

So here's an unobtrusive hack that works: (less disruptive to the file then changing variable names)

# allow ncvis to view pg2 variables:
ncatted -a standard_name,lon_gll,o,c,"MISSING" *.h.AVERAGE*.nc
ncatted -a standard_name,lat_gll,o,c,"MISSING" *.h.AVERAGE*.nc
ncatted -a standard_name,lon_pg2,o,c,"longitude" *.h.AVERAGE*.nc
ncatted -a standard_name,lat_pg2,o,c,"latitude" *.h.AVERAGE*.nc

# allow ncvis to view gll variables:
ncatted -a standard_name,lon_pg2,o,c,"MISSING" *.h.AVERAGE*.nc
ncatted -a standard_name,lat_pg2,o,c,"MISSING" *.h.AVERAGE*.nc
ncatted -a standard_name,lon_gll,o,c,"longitude" *.h.AVERAGE*.nc
ncatted -a standard_name,lat_gll,o,c,"latitude" *.h.AVERAGE*.nc