.NET 6 #763

Kinematics · 2021-11-20T18:28:06Z

Kinematics
Nov 20, 2021

Now that .NET 6 was officially released, along with Visual Studio 2022 as the only version of VS that officially supports it, I've been running benchmarks comparing performance with the previous .NET 5 builds.

The only changes I've made have been the target version of .NET. All the code remains the same.

Some algorithms don't seem to change at all (mostly the simplest algorithms). A few seem to gain about 2%-5% in performance. And a few have suspiciously large gains. This last case seems to be an effect which causes the .NET 5 tests to have substantially large variations for no reason that I can immediately identify.

One benchmark (using BenchmarkDotNet) varied between 820 us and 895 us in .NET 5, while the .NET 6 version was 753 us, so the gain could be between 8% and 16%.

Running the loop count saw one algorithm have the highest gain, at about an 8% increase in the number of loops, but it was also the most complex algorithm ([8 of 30] with bitmasking).

Anyway, individual scores seem to vary by up to 4% per run, which is problematic when comparing tests. I feel comfortable saying that a typical algorithm is likely to improve by ~3% (but may not improve at all), but anything beyond that shouldn't be trusted unless you did some major rewriting to take advantage of new .NET 6 tools.

BenchmarkDotNet comparison numbers

.NET 5

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.19042.1348 (20H2/October2020Update)
AMD Ryzen 7 3700X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=6.0.100
  [Host]     : .NET 5.0.12 (5.0.1221.52207), X64 RyuJIT
  DefaultJob : .NET 5.0.12 (5.0.1221.52207), X64 RyuJIT

Method	SieveSize	Mean	Error	StdDev	Ratio	Gen 0	Gen 1	Gen 2	Allocated
Original	1000000	1,345.9 us	4.22 us	3.95 us	0.98	5.8594	-	-	62,592 B
Standard	1000000	1,369.5 us	6.82 us	6.38 us	1.00	5.8594	-	-	62,592 B
Bool	1000000	743.5 us	2.31 us	2.16 us	0.54	142.5781	142.5781	142.5781	500,056 B
InvertedBool	1000000	694.2 us	3.41 us	3.19 us	0.51	142.5781	142.5781	142.5781	500,056 B
DirectInvertedBool	1000000	693.5 us	4.25 us	3.98 us	0.51	142.5781	142.5781	142.5781	500,056 B
RawBits	1000000	814.5 us	3.78 us	3.54 us	0.59	19.5313	19.5313	19.5313	62,560 B
ArrayPoolClass	1000000	819.9 us	5.10 us	4.77 us	0.60	-	-	-	32 B
ArrayPool2Of6	1000000	512.5 us	1.77 us	1.66 us	0.37	-	-	-	32 B
ArrayPool6Par	1000000	716.8 us	4.44 us	3.94 us	0.52	35.1563	-	-	295,988 B
ArrayPool8of30	1000000	400.5 us	0.78 us	0.73 us	0.29	-	-	-	96 B
ArrayPool8of30M	1000000	311.8 us	1.52 us	1.42 us	0.23	-	-	-	912 B

.NET 6

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.19042.1348 (20H2/October2020Update)
AMD Ryzen 7 3700X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=6.0.100
  [Host]     : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
  DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT

Method	SieveSize	Mean	Error	StdDev	Ratio	Gen 0	Gen 1	Gen 2	Allocated
Original	1000000	1,339.4 us	1.84 us	1.44 us	1.00	5.8594	-	-	62,593 B
Standard	1000000	1,342.4 us	1.21 us	1.01 us	1.00	5.8594	-	-	62,593 B
Bool	1000000	743.4 us	1.76 us	1.65 us	0.55	142.5781	142.5781	142.5781	500,105 B
InvertedBool	1000000	556.4 us	2.37 us	2.22 us	0.41	142.5781	142.5781	142.5781	500,105 B
DirectInvertedBool	1000000	556.0 us	1.72 us	1.61 us	0.41	142.5781	142.5781	142.5781	500,105 B
RawBits	1000000	778.1 us	2.30 us	2.15 us	0.58	19.5313	19.5313	19.5313	62,567 B
ArrayPoolClass	1000000	752.6 us	2.63 us	2.33 us	0.56	-	-	-	33 B
ArrayPool2Of6	1000000	529.4 us	3.19 us	2.99 us	0.39	-	-	-	33 B
ArrayPool6Par	1000000	676.1 us	2.48 us	2.32 us	0.50	36.1328	-	-	301,306 B
ArrayPool8of30	1000000	387.2 us	1.98 us	1.75 us	0.29	-	-	-	96 B
ArrayPool8of30M	1000000	288.2 us	1.30 us	1.22 us	0.21	-	-	-	912 B

Latest run data

NET 6 in VS 2022

Notable differences from Readme.md (test from several months ago using NET 5 in VS 2019), to NET 5 in VS2019, to NET 5 in VS 2022, to NET 6 in VS 2022:

bool array: 6643 → 5756 → 5652 → 6285
inverted bool array: 8800 → 7274 → 7107 → 8926
direct access inverted bool array: 8964 → 7247 → 7110 → 8928
array pool: 6084 → 6096 → 6034 → 6658
array pool [2 of 6]: 9670 → 9882 → 9528 → 9468
array pool [8 of 30]: 12467 → 12421 → 12265 → 12821
array pool [8 of 30] with bitmasking: 15611 → 15643 → 15817 → 17308

What I see is that several tests are running slower now than they were a few months ago, but compared to .NET 5, .NET 6 is running faster.

There's only two cases where it's clearly faster in .NET 6 than in any .NET 5 run: 1) the basic array pool version; and 2) the [8 of 30] array pool using bitmasking — rather more complicated code than the other algorithms.

The bool array algorithms also run faster than the .NET 5 versions, but only on par with the older tests, which makes me a bit uncertain on whether this is a real performance gain or not.

@Kinematics: Starting (original)...
Passes: 3545, Time: 5.00124 s, Per Loop: 1.410719 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_original;3545;5.00124;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (standard)...
Passes: 3687, Time: 5.00099 s, Per Loop: 1.356116 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_standard;3687;5.00099;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (bool array)...
Passes: 6285, Time: 5.00022 s, Per Loop: 0.795545 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_bool;6285;5.00022;1;algorithm=base,faithful=yes,bits=8

@Kinematics: Starting (inverted bool array)...
Passes: 8926, Time: 5.00009 s, Per Loop: 0.560161 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_ibool;8926;5.00009;1;algorithm=base,faithful=yes,bits=8

@Kinematics: Starting (direct access inverted bool array)...
Passes: 8928, Time: 5.00052 s, Per Loop: 0.560036 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_dbool;8928;5.00052;1;algorithm=base,faithful=yes,bits=8

@Kinematics: Starting (raw bits)...
Passes: 6449, Time: 5.00052 s, Per Loop: 0.775314 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw;6449;5.00052;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (raw bits uint)...
Passes: 6616, Time: 5.00009 s, Per Loop: 0.755744 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw32;6616;5.00009;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (raw bits direct)...
Passes: 6454, Time: 5.00025 s, Per Loop: 0.774713 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_rawd;6454;5.00025;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (raw bits [2 of 6])...
Passes: 6125, Time: 5.00021 s, Per Loop: 0.816327 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw6;6125;5.00021;1;algorithm=wheel,faithful=yes,bits=1

@Kinematics: Starting (raw bits parallel version)...
Passes: 4366, Time: 5.00071 s, Per Loop: 1.145213 ms, Sieve Size: 1000000, Thread Count: 1, Parallel Thread Count: 16, Primes Found: 78498, Valid: True
kinematics_rawp;4366;5.00071;16;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (array pool)...
Passes: 6658, Time: 5.00036 s, Per Loop: 0.750976 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool;6658;5.00036;1;algorithm=base,faithful=yes,bits=1

@Kinematics: Starting (array pool [2 of 6])...
Passes: 9468, Time: 5.00034 s, Per Loop: 0.528095 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool2of6;9468;5.00034;1;algorithm=wheel,faithful=yes,bits=1

@Kinematics: Starting (parallel array pool [2 of 6])...
Passes: 5531, Time: 5.0002 s, Per Loop: 0.903996 ms, Sieve Size: 1000000, Thread Count: 1, Parallel Thread Count: 16, Primes Found: 78498, Valid: True
kinematics_pool6p;5531;5.0002;16;algorithm=wheel,faithful=yes,bits=1

@Kinematics: Starting (array pool [8 of 30])...
Passes: 12821, Time: 5.0002 s, Per Loop: 0.389985 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool30;12821;5.0002;1;algorithm=wheel,faithful=yes,bits=1

@Kinematics: Starting (array pool [8 of 30] with bitmasking)...
Passes: 17308, Time: 5.00002 s, Per Loop: 0.288884 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool30m;17308;5.00002;1;algorithm=wheel,faithful=yes,bits=1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.NET 6 #763

{{title}}

.NET 5

.NET 6

NET 6 in VS 2022

Replies: 0 comments

Select a reply

.NET 6 #763

Kinematics Nov 20, 2021

.NET 5

.NET 6

NET 6 in VS 2022

Replies: 0 comments

Kinematics
Nov 20, 2021