Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IterationSetup allocates 400B #2582

Open
AleksLigeza opened this issue Jun 22, 2024 · 1 comment · May be fixed by #2562
Open

IterationSetup allocates 400B #2582

AleksLigeza opened this issue Jun 22, 2024 · 1 comment · May be fixed by #2562
Assignees

Comments

@AleksLigeza
Copy link

Hello!
I would like to report that using [IterationSetup] is causing 400B to be allocated.
I quickly checked how it works for earlier versions and it seems that the problem started in 0.13.6 and persists in 0.13.12.

Please check the summary and code below:

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4529/22H2/2022Update)
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 8.0.106
[Host] : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-HVPPIJ : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

InvocationCount=1 UnrollFactor=1

Method Mean Error Allocated
TestBenchmark 0.0 ns 0.0 ns 400 B
using BenchmarkDotNet.Attributes;

BenchmarkDotNet.Running.BenchmarkRunner.Run<Test>();

[MemoryDiagnoser]
public class Test
{
    [IterationSetup]
    public void TestSetup()
    {
    }

    [Benchmark]
    public void TestBenchmark()
    {
    }
}

If I remove IterationSetup or change it to GlobalSetup, the problem does not occur.

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4529/22H2/2022Update)
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK 8.0.106
[Host] : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
DefaultJob : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Method Mean Error StdDev Allocated
TestBenchmark 0.0698 ns 0.0279 ns 0.0466 ns -

Thanks!

@timcassell
Copy link
Collaborator

This is because [IterationSetup] causes the benchmark method to be invoked only once. Removing it allows it to be invoked multiple times. There are background allocations that we cannot control (see dotnet/runtime#101536), and more benchmark invocations drowns those out.

#2562 will improve it somewhat, but will not completely eliminate the issue. The runtime will need to remove the background allocations to completely fix it, and that's out of our control.

@timcassell timcassell linked a pull request Jun 22, 2024 that will close this issue
@timcassell timcassell self-assigned this Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants