Problem after upgrade ("depth dimension has the wrong format. It should be [zdim, ydim, xdim]'")

[ana-machado]

Hello!
I needed to upgrade my python installation, after that I also needed to install the latest Parcels version and my code stopped working. I was already able to change some of the things, but I am not being able to solve this error. I am using ROMS outputs, in which I created a variable calles SigmaDepths that has the depths of the sigma coordinates. It was defined like this:
filenames = {'U': {'lon': grid_file, 'lat': grid_file, 'depth': data_file, 'data': data_file},
'V': {'lon': grid_file, 'lat': grid_file, 'depth': data_file, 'data': data_file},
'T': {'lon': grid_file, 'lat': grid_file, 'depth': data_file, 'data': data_file},
'RHO': {'lon': grid_file, 'lat': grid_file, 'depth': data_file, 'data': data_file},
'HBAT': {'lon': grid_file, 'lat': grid_file,'data': data_file},
'CHLA': {'lon': chla_file, 'lat': chla_file, 'data': chla_file}}

variables = {'U': 'u', 'V': 'v', 'T': 'temp', 'HBAT': 'h', 'CHLA': 'CHL'}

dimensions = { 'U': {'lon': 'lon_psi', 'lat': 'lat_psi', 'depth': 'SigmaDepths', 'time': 'roms_time2'},
'V': {'lon': 'lon_psi', 'lat': 'lat_psi', 'depth': 'SigmaDepths', 'time': 'roms_time2'},
'T': {'lon': 'lon_rho', 'lat': 'lat_rho', 'depth': 'SigmaDepths', 'time': 'roms_time2'},
'HBAT': {'lon': 'lon_rho', 'lat': 'lat_rho'},
'CHLA': {'lon': 'lon', 'lat': 'lat', 'time': 'chla_time2'}}

fieldset = FieldSet.from_c_grid_dataset(filenames, variables, dimensions)

It was working perfectly, but now I am getting the errror copy-pasted below.
Can someone please help me? I have some urgency to do these simulations, and cannot progress till I find a solution for this problem.

Thank you very much.

Traceback (most recent call last):
File "/data/ParcelsSardines/run_parcels_anchovie_2016/runParcelsAnchovieIBM.py", line 57, in
fieldset = FieldSet.from_c_grid_dataset(filenames, variables, dimensions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 673, in from_c_grid_dataset
return cls.from_netcdf(filenames, variables, dimensions, mesh=mesh, indices=indices, time_periodic=time_periodic,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 454, in from_netcdf
fields[var] = Field.from_netcdf(paths, (var, name), dims, inds, grid=grid, mesh=mesh, timestamps=timestamps,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/field.py", line 467, in from_netcdf
grid = Grid.create_grid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 80, in create_grid
return RectilinearSGrid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 386, in init
assert self.xdim == self.depth.shape[-1], 'depth dimension has the wrong format. It should be [zdim, ydim, xdim]'
AssertionError: depth dimension has the wrong format. It should be [zdim, ydim, xdim]
Traceback (most recent call last):
File "/data/ParcelsSardines/run_parcels_anchovie_2016/runParcelsAnchovieIBM.py", line 56, in
fieldset = FieldSet.from_c_grid_dataset(filenames, variables, dimensions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 673, in from_c_grid_dataset
return cls.from_netcdf(filenames, variables, dimensions, mesh=mesh, indices=indices, time_periodic=time_periodic,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 454, in from_netcdf
fields[var] = Field.from_netcdf(paths, (var, name), dims, inds, grid=grid, mesh=mesh, timestamps=timestamps,

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/field.py", line 467, in from_netcdf
grid = Grid.create_grid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 80, in create_grid
return RectilinearSGrid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 386, in init
assert self.xdim == self.depth.shape[-1], 'depth dimension has the wrong format. It should be [zdim, ydim, xdim]'
AssertionError: depth dimension has the wrong format. It should be [zdim, ydim, xdim]
Traceback (most recent call last):
File "/data/ParcelsSardines/run_parcels_anchovie_2016/runParcelsAnchovieIBM.py", line 56, in
fieldset = FieldSet.from_c_grid_dataset(filenames, variables, dimensions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 673, in from_c_grid_dataset
return cls.from_netcdf(filenames, variables, dimensions, mesh=mesh, indices=indices, time_periodic=time_periodic,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/fieldset.py", line 454, in from_netcdf
fields[var] = Field.from_netcdf(paths, (var, name), dims, inds, grid=grid, mesh=mesh, timestamps=timestamps,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/field.py", line 467, in from_netcdf
grid = Grid.create_grid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 80, in create_grid
return RectilinearSGrid(lon, lat, depth, time, time_origin=time_origin, mesh=mesh, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ipma/miniconda3/envs/parcels/lib/python3.12/site-packages/parcels/grid.py", line 386, in init
assert self.xdim == self.depth.shape[-1], 'depth dimension has the wrong format. It should be [zdim, ydim, xdim]'
AssertionError: depth dimension has the wrong format. It should be [zdim, ydim, xdim]
Traceback (most recent call last):

[erikvansebille]

Hi @ana-machado; sorry that you're experiencing this problem. I don't immediately see what's wrong, and unfortunately I don't have time now to delve into it too deeply the coming weeks. I understand you're in a rush; so perhaps you could 'downgrade' your Parcels version to the version that worked before? In conda you can select which version to install

[michaeldenes]

This looks eerily similar to #1497, maybe try the solution found there, or as Erik suggests (and the approach I would take if time is tight) install a previous version of Parcels that doesn't throw this error.

[ana-machado]

Hello Eric and Michael,
Thank you both for your replies. I followed Erik advice, downgraded the Parcels installation and the code is running again. I think the problem comes from this function create_grid, it seems to me that before, the grid was provided directly, and now it is expected to create the grid automatically but something failed.
Now I was in a hurry to finish these simulations, I'll try to upgrade again later and test Michael suggestion, and try to understand what might be happening.