Incorrect memory allocation measurements due to global variables

[huiyuxie]

This issue was identified due to the unusual memory allocation behavior observed in #2150. It was found that the memory allocation bytes differ greatly between the first and second runs of the rhs! function.

I further went into this issue and found there may be a problem with how we measure the memory allocation using allocated macro - in short, it is suggested to avoid using global variables to achieve same memory allocation bytes across infinite runs (i.e., all runs produce the same result).

But here the developer is doing something wrong, for example

Trixi.jl/test/test_dgmulti_1d.jl

Lines 17 to 30 in 9c2f326

	@trixi_testset "elixir_advection_gauss_sbp.jl " begin
	@test_trixi_include(joinpath(EXAMPLES_DIR, "elixir_advection_gauss_sbp.jl"),
	cells_per_dimension=(8,),
	l2=[2.9953644500009865e-5],
	linf=[4.467840577382365e-5])
	# Ensure that we do not have excessive memory allocations
	# (e.g., from type instabilities)
	let
	t = sol.t[end]
	u_ode = sol.u[end]
	du_ode = similar(u_ode)
	@test (@allocated Trixi.rhs!(du_ode, u_ode, semi, t)) < 1000
	end
	end

The first run of rhs! is included in the file elixir_advection_gauss_sbp.jl and the second run of rhs! is inside let block, If we further examine the second run of rhs!, it actually reports zero allocation bytes when we print it. This unusual behavior is caused by the use of global variables in the file we ran initially - at least semi is global during this module. So we never captured the real memory allocation throughout all the tests in this repository.

You can also double check here about how the introduction of global var will influence the memory allocation.
The first example -

# Define the global variable
global myarray = fill(1.0, 10^6)  # Create a global array of 1 million elements, all set to 1.0

# Function uses the global array
function myfunc(array)
    sum(array)  # Compute the sum of the array
end

# First run
allocation1 = @allocated myfunc(myarray)  

# Second run
allocation2 = @allocated myfunc(myarray)  

println("First run: $allocation1 bytes")
println("Second run: $allocation2 bytes")

and

# No global variables
function myfunc()
    x = fill(1.0, 10^6)  # Create an array of 1 million elements, all set to 1.0
    sum(x)               # Compute the sum
end

# First run
allocation1 = @allocated myfunc()  

# Second run
allocation2 = @allocated myfunc()  

println("First run: $allocation1 bytes")
println("Second run: $allocation2 bytes")

The second example -

function foo(c)
    v = c * 1
    v
end

# Define global variables
c = 1.0 

function test_allocations()
    println(@allocated foo(c))
    println(@allocated foo(c))
    println(@allocated foo(c))
end

test_allocations()

# No global variables
function foo(c)
    v = c * 1
    v
end

function test_allocations()
    c = 1.0  # Local variable
    println(@allocated foo(c))
    println(@allocated foo(c))
    println(@allocated foo(c))
end

test_allocations()

xref https://discourse.julialang.org/t/using-allocated-to-track-memory-allocations/4617

cc @ranocha @jlchan @DanielDoehring

[huiyuxie]

Please take a look at it and address this first if you have time thanks

[ranocha]

Is the difference coming from potentially compiling the function at the first call and including the corresponding allocations or not? For example, the docstring of @time states:

help?> @time
  @time expr
  @time "description" expr

  [...]

  In some cases the system will look inside the @time expression and compile some of the called code
  before execution of the top-level expression begins. When that happens, some compilation time will
  not be counted. To include this time you can run @time @eval ....

  [...]

What would be the preferred way to check whether executing a function leads to an undesirable large amount of allocations, @vchuravy ?

[vchuravy]

@ranocha yes this is correct. In type-stable code the compiler allocations are sometimes hidden.

E.g., in the second example the version without a global variable for me shows zero allocations even on the first call, whereas with an access to a global variable the first call allocates a lot and every subsequent call allocates some.

One could use a manual precompile(f, (Float64.)) to avoid the compilation allocations, but that only works until you hit a type-instability.

Generally? Execute twice, measure the second time.

[huiyuxie]

In type-stable code the compiler allocations are sometimes hidden.

Why is it sometimes hidden, and what are you referring to when you say sometimes (i.e., in which cases)? @vchuravy

Execute twice, measure the second time.

I think that is what you/we are currently doing here, right? One thing I want to confirm is whether a memory allocation print of zero in the case of elixir_advection_gauss_sbp.jl (which I mentioned) is a normal result for you. @ranocha

[ranocha]

Yes, having zero allocations in rhs! is exactly what we want to have (except that a few allocations are required/acceptable in multi-threaded execution).

[ranocha]

Since we execute rhs! already in the tests before, this issue can be closed, can't it?

[huiyuxie]

I have no idea - the real problem is that #2150 has to run @allocated three times to reduce memory allocation to less than 5,000. I'm just helping to find out the reasons behind here

[huiyuxie]

I just checked the memory allocation bytes in #2150 and found that the results in the second run and third run are both zeros - it is not what I was informed.

cc @jlchan Could you please check again? I could not find any problem with memory allocation in your PR.

[huiyuxie]

@ranocha Yes this issue can be closed 😊