Question about FillPatcherFill usage in amrLevel

[Afraid31]

Hi everyone
I'm working on an amrLevel computation where I need to handle ghost cell filling for fine levels (lev > 0) with specific requirements:

• I want to preserve the ghost cell data of the first layer of fine grids (only the outermost 1 layer of ghost cells).
• For all other ghost cell layers (beyond the first layer), I need to interpolate and fill them using data from lev-1 via FillPatcherFill.

What I've tried:

FillPatcherFill(Moment_border, 0, NUM_MOMENT, NUM_GROW, time, Moment_Type, 0);
Moment_border.LocalCopy(Moment_old, 0, 0, NUM_MOMENT, IntVect(1));

This worked only when the BoxArray had a single box. In multi-box cases, a small portion of the first-layer ghost cells (which should be preserved) were incorrectly overwritten by interpolation.

I adjusted the grid range before/after FillPatcherFill to avoid overwriting the first ghost layer, and this fixed the multi-box issue on CPU (Windows, non-parallel). Notably, I can ensure that the data in the overlapping valid grids is consistent after grids.grow(1).

grids = grids.grow(1);
FillPatcherFill(Moment_border, 0, NUM_MOMENT, NUM_GROW - 1, time, Moment_Type, 0);
grids = grids.grow(-1);

My core questions:
1.Is this revised approach valid and safe for GPU parallelization (e.g., CUDA)? I'm concerned about potential data consistency or memory issues in GPU mode.
2.Can this approach also be applied to the regrid process if I want to preserve the first-layer ghost cells of fine grids during regridding?

[WeiqunZhang]

Your approach is not recommended because FillPath in general depends on that the fine grids are coarsenable.

Did you try my suggestion in #4873? #4873 (comment)
Let me elaborate on it. You can do something like below. It should also work for regrid.

FillPath.... // as normal;
Moment_old.FillBoundary(0, NUM_MOMENT, IntVect(1), this_level_s_geom.periodicity());
Moment_border.ParallelCopy(Moment_old, 0, 0, NUM_MOMENT, IntVect(1), IntVect(1), this_level_s_geom.periodicity());

[Afraid31]

I tried your method and it does work effectively! It seems that there are some incomplete considerations near the periodic boundaries, but this method is already sufficient to handle my current use case. Thank you very much for your help!