CuGPU not defined error

[kodingkoning]

I am trying to change my simulation to use GPUs rather than just CPUs. I changed sim!(output, spec_rule,nbr_rule; proc = ThreadedCPU(), boundary=Wrap()) to use "CuGPU" instead of "ThreadedCPU", but then I get the error: nested task error: UndefVarError: CuGPU not defined

What other changed are required for CuGPU() to be recognized?

[rafaqz]

Did you try using CUDA first ?

[kodingkoning]

Yes, this is with using CUDA

[rafaqz]

Ok may need DynamicGrids.CuGPU

[kodingkoning]

I just tried DynamicGrids.CuCPU() as well.

I simplified it to the simplest code that gives me the error, which is:

using DynamicGrids, CUDA

DynamicGrids.CuGPU()

and results in:

ERROR: LoadError: UndefVarError: CuGPU not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:35
 [2] top-level scope
   @ ~/julia_code/test_cuda.jl:3
in expression starting at ~/julia_code/test_cuda.jl:3

The same thing happens with either DynamicGrids.CuGPU() or CuGPU() alone

[ederag]

If you are on julia-1.10.2, then I could reproduce, and it seems to require the same workaround as in #243:

(@v1.10) pkg> activate --temp
  Activating new project at `/tmp/jl_7aYPFi`

(jl_G6yIEw) pkg> add DynamicGrids#dev, CUDA

julia> DynamicGrids.CuGPU()
CuGPU{32}()

[ederag]

But if you are trying the README, you'll then face

julia> using CUDAKernels, CUDA
 │ Package CUDAKernels not found, but a package named CUDAKernels is available from a registry. 
 │ Install package?
 │   (jl_G6yIEw) pkg> add CUDAKernels 
 └ (y/n/o) [y]: 
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package CUDAKernels [72cfdca4]:
 CUDAKernels [72cfdca4] log:
 ├─possible versions are: 0.1.0-0.4.7 or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions: 0.1.0-0.4.7
 └─restricted by compatibility requirements with KernelAbstractions [63c18a36] to versions: uninstalled — no versions left
   └─KernelAbstractions [63c18a36] log:
     ├─possible versions are: 0.1.0-0.9.18 or uninstalled
     └─restricted to versions 0.9 by DynamicGrids [a5dba43e], leaving only versions: 0.9.0-0.9.18
       └─DynamicGrids [a5dba43e] log:
         ├─possible versions are: 0.21.4 or uninstalled
         └─DynamicGrids [a5dba43e] is fixed to version 0.21.4

[rafaqz]

We should really release a new version. The problem is its a breaking version and I'm not sure all the breaks make sense, and there are likely a few more on the way.

But I have been working on branches for over a year now, dev is up-to-date and actively worked on.

So using DynamicGrids#dev is highly reccomended.

[ederag]

Indeed, the breaking changes made me give up for now, as I just wanted to show your awesome package to a colleague.
Thanks to julia Manifest system, it was as simple as reopening the old notebook with julia 1.9.2.

dev is up-to-date and actively worked on.

Nice, and just to be clear, #244 (comment) was with DynamicGrids#dev and julia 1.10.2.

[rafaqz]

Yeah... my aim is to get work back on main and registered soon. But my other more general packages have been taking most of my time, plus PhD constraints.

I actually use this constantly in all of my work, but not in a stable-releases kind of way...

[kodingkoning]

Thank you! I had been using an older version of Julia (1.7.3) because it was available on the cluster I was using, but I installed 1.10.2 myself, and used DynamicGrids#dev, and that's cleared up the error.

[kodingkoning]

Reopening because I thought my further errors were unrelated, but now I'm thinking they are related. If it's not related, I'm glad to separate this out. Any insight on this? Am I missing some change I need to make to use the GPU? All I have changed is the value of proc to CuGPU(), and of course adding using CUDA, DynamicGrids#dev.

I'm using this code, which works with DynamicGrids and DynamicGrids#dev on the CPU, but not with the GPU.

Full error:

ERROR: LoadError: InvalidIRError: compiling MethodInstance for DynamicGrids.gpu_ka_rule_kernel!(::KernelAbstractions.CompilerMetadata{KernelAbstractions.NDIteration.StaticSize{(100, 100)}, KernelAbstractions.NDIteration.DynamicCheck, Nothing, Nothing, KernelAbstractions.NDIteration.NDRange{2, KernelAbstractions.NDIteration.StaticSize{(4, 4)}, KernelAbstractions.NDIteration.StaticSize{(32, 32)}, Nothing, Nothing}}, ::DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuDeviceMatrix{Float64, 1}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuDeviceMatrix{Float64, 1}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, ::Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, ::Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, ::Val{:_default_}, ::Val{:_default_}) resulted in invalid LLVM IR
Reason: unsupported call to an unknown function (call to julia.new_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.push_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Stacktrace:
 [1] Array
   @ ./boot.jl:477
 [2] vect
   @ ./array.jl:163
 [3] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:41
 [4] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [5] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [6] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [7] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [8] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.get_gc_frame_slot)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to *)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to +)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to /)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to *)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:48
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to <=)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:48
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.pop_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Hint: catch this exception as `err` and call `code_typed(err; interactive = true)` to introspect the erronous code with Cthulhu.jl
Stacktrace:
  [1] check_ir(job::GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, args::LLVM.Module)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/validation.jl:147
  [2] macro expansion
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:440 [inlined]
  [3] macro expansion
    @ ~/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:253 [inlined]
  [4] macro expansion
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:439 [inlined]
  [5] emit_llvm(job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, only_entry::Bool, validate::Bool)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:92
  [6] emit_llvm
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:86 [inlined]
  [7] codegen(output::Symbol, job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:129
  [8] codegen
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:110 [inlined]
  [9] compile(target::Symbol, job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, strip::Bool, validate::Bool, only_entry::Bool)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:106
 [10] compile
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:98 [inlined]
 [11] #1075
    @ ~/.julia/packages/CUDA/rXson/src/compiler/compilation.jl:247 [inlined]
 [12] JuliaContext(f::CUDA.var"#1075#1077"{GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}})
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:47
 [13] compile(job::GPUCompiler.CompilerJob)
    @ CUDA ~/.julia/packages/CUDA/rXson/src/compiler/compilation.jl:246
 [14] actual_compilation(cache::Dict{Any, CuFunction}, src::Core.MethodInstance, world::UInt64, cfg::GPUCompiler.CompilerConfig{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, compiler::typeof(CUDA.compile), linker::typeof(CUDA.link))
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:125
 [15] cached_compilation(cache::Dict{Any, CuFunction}, src::Core.MethodInstance, cfg::GPUCompiler.CompilerConfig{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, compiler::Function, linker::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:103
 [16] macro expansion
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:359 [inlined]
 [17] macro expansion
    @ ./lock.jl:267 [inlined]
 [18] cufunction(f::typeof(DynamicGrids.gpu_ka_rule_kernel!), tt::Type{Tuple{KernelAbstractions.CompilerMetadata{KernelAbstractions.NDIteration.StaticSize{(100, 100)}, KernelAbstractions.NDIteration.DynamicCheck, Nothing, Nothing, KernelAbstractions.NDIteration.NDRange{2, KernelAbstractions.NDIteration.StaticSize{(4, 4)}, KernelAbstractions.NDIteration.StaticSize{(32, 32)}, Nothing, Nothing}}, DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuDeviceMatrix{Float64, 1}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuDeviceMatrix{Float64, 1}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Val{:_default_}, Val{:_default_}}}; kwargs::@Kwargs{always_inline::Bool, maxthreads::Int64})
    @ CUDA ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:354
 [19] cufunction
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:351 [inlined]
 [20] macro expansion
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:104 [inlined]
 [21] #_#6
    @ ~/.julia/packages/CUDA/rXson/src/CUDAKernels.jl:118 [inlined]
 [22] Kernel
    @ ~/.julia/packages/CUDA/rXson/src/CUDAKernels.jl:104 [inlined]
 [23] maprule!(data::DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, proc::CuGPU{32}, opt::NoOpt, ruletype::Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, rule::Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, rkeys::Val{:_default_}, wkeys::Val{:_default_})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:48
 [24] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:65 [inlined]
 [25] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:17 [inlined]
 [26] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:10 [inlined]
 [27] sequencerules!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/sequencerules.jl:16 [inlined]
 [28] sequencerules!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/sequencerules.jl:5 [inlined]
 [29] |>
    @ ./operators.jl:917 [inlined]
 [30] _step!(sd::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, rules::Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:208
 [31] simloop!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, simdata::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, Matrix{Float64}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::Matrix{Float64}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, fspan::UnitRange{Int64}; printframe::Bool)
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:187
 [32] simloop!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:171 [inlined]
 [33] #runsim!#138
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:160 [inlined]
 [34] runsim!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, simdata::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, Matrix{Float64}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::Matrix{Float64}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, fspan::UnitRange{Int64})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:156
 [35] sim!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}; init::Matrix{Float64}, mask::Nothing, tspan::UnitRange{Int64}, aux::Nothing, fps::Nothing, boundary::Wrap, proc::CuGPU{32}, opt::NoOpt, cellsize::Int64, timestep::Nothing, simdata::Nothing, kw::@Kwargs{})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:97
 [36] #sim!#136
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:100 [inlined]
 [37] simulation(nrep::Int64, step::Int64)
    @ Main ~/stochastic_competition_git/julia_code/sim_git.jl:92
 [38] func(nsteps::Int64, nrep::Int64)
    @ Main ~/stochastic_competition_git/julia_code/sim_git.jl:106
 [39] macro expansion
    @ ~/stochastic_competition_git/julia_code/sim_git.jl:111 [inlined]
 [40] macro expansion
    @ ./timing.jl:279 [inlined]
 [41] top-level scope
    @ ~/stochastic_competition_git/julia_code/sim_git.jl:269
in expression starting at /u/ekoning2/stochastic_competition_git/julia_code/sim_git.jl:110

[rafaqz]

Ok. So first make sure your model runs fine with e.g. proc=CPUGPU().

Then you need to be 100% sure all of the code inside your applyrule functions or rule closures are type stable and non-mutating/non-allocating. This is a GPU problem generally, rather than a DynamicGrids.jl problem - unstable or allocating code simply cant work on a GPU.

Those errors all suggest that things like this are happening. You really dont want to allocate inside applyrule even on CPU, its a huge performance hit. You will notice DynamicGrids.jl itself allocates essentially nothing over a whole simulation for this reason (e.g. using Life()). Likely fixing the allocations and type stability will mean you don't need a GPU at all, and your simulation is plenty fast on your desktop CPU!

To fix this I suggest running your CPU simulations for a few seconds using ProfileView.jl @profview.

Any red (instability) or yellow (allocations) patches in the inner function calls will mean that your model wont run on a GPU. Cthulhu.jl can help, you can click on the ProfileView.jl profile and call descend_clicked() to explore the types in your functions.

There is a method isinferred in the dev branch that will tell you if types are inferred for your rule.

As for the cluster, get your manager to add 1.10, its much better :)

(honestly here I never end up using a cluster because with no allocations my desktop is happy to run my sims a million times!)

[rafaqz]

If you paste your rules here I can probably tell you what wrong

Oops you had linked it.

This is certainly a problem, allocating an Array here for ever cell at every timestep:

upordown=sample([1,-1])

use StaticArrays.jl, and:

sample(@SVector[1, -1])

As is:

collect(1:8)

Just use 1:8 !

Basically, you should never allocate or write to any array inside a rule other than writing to a grid with e.g. add! inside a SetRule. Mostly this isn't a problem - just use StaticArrays or Tuples instead, and functional approaches like map/reduce rather than loops that write to arrays.