Performance regression in AVX2 instrinsic-heavy code after stdarch subtree update

[fintelia]

Code

I tried this code: simd-adler32 crate's AVX2 implementation on a Zen 5 processor.

I expected to see this happen: Throughput of 65+ GiB/s

$ cargo +nightly-2025-11-03 bench --bench variants -- --warm-up-time=1 --measurement-time=3 variants/avx2-100k
variants/avx2-100k      time:   [1.3893 µs 1.3898 µs 1.3907 µs]                                
                        thrpt:  [66.969 GiB/s 67.010 GiB/s 67.038 GiB/s]
                 change:
                        time:   [-0.1708% -0.0821% +0.0269%] (p = 0.14 > 0.05)
                        thrpt:  [-0.0269% +0.0822% +0.1711%]
                        No change in performance detected.

Instead, this happened: Throughput regressed to under 40 GiB/s

$ cargo +nightly-2025-11-04 bench --bench variants -- --warm-up-time=1 --measurement-time=3 variants/avx2-100k
variants/avx2-100k      time:   [2.3893 µs 2.3918 µs 2.3949 µs]                                
                        thrpt:  [38.888 GiB/s 38.938 GiB/s 38.979 GiB/s]
                 change:
                        time:   [+71.687% +71.923% +72.125%] (p = 0.00 < 0.05)
                        thrpt:  [-41.903% -41.834% -41.755%]
                        Performance has regressed.

Version it worked on

It most recently worked on: Rust 1.92.0 and nightly-2025-11-03

Version with regression

rustc --version --verbose:

rustc +nightly-2025-11-04 --version --verbose
rustc 1.93.0-nightly (20383c9f1 2025-11-03)
binary: rustc
commit-hash: 20383c9f1d84eb9b9c6668a1668ef68a81eae274
commit-date: 2025-11-03
host: x86_64-unknown-linux-gnu
release: 1.93.0-nightly
LLVM version: 21.1.3

Bisection details

The PR introducing the regression in this rollup is #148402: stdarch subtree update

searched nightlies: from nightly-2025-11-03 to nightly-2025-11-04
regressed nightly: nightly-2025-11-04
searched commit range: b15a874...20383c9
regressed commit: f2bae99

bisected with cargo-bisect-rustc v0.6.11

Host triple: x86_64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --start=2025-11-03 --end=2025-11-04 --script bisect.sh 

bisect.sh:

#!/bin/bash
CARGO_TARGET_DIR=target cargo bench --bench variants -- --warm-up-time=1 --noplot --baseline stable --measurement-time=3 variants/avx2-100k | rg "No change in performance detected|Change within noise threshold."

@rustbot modify labels: +regression-from-stable-to-beta -regression-untriaged

[usamoi]

Caused by rust-lang/stdarch@575823d. LLVM fails to select vpmaddwd for _mm256_madd_epi16. @sayantn

[folkertdev]

LLVM is able to select that instruction just fine, provided that the requisite target features are available:

https://godbolt.org/z/a3TGv8TqG

The simd-adler32 function that uses the intrinsics does not have avx2 available. It sort of worked before because ultimately I assume everything is inlined into update_imp which does have the target feature. But for the implementation to be correct, all of the helper functions that (indirectly) use avx2 intrinsics also need to declare #[target_feature(enable = "avx2")].

https://github.com/mcountryman/simd-adler32/blob/dc6439879ef4f6b88f5b5a53f2e789cfa7842a6c/src/imp/avx2.rs#L107

[usamoi]

LLVM is able to select vpmaddwd for _mm256_madd_epi16, but not for reduce_add_blocks.

https://godbolt.org/z/f131v78or

If you change _mm256_madd_epi16 to _mm256_madd_epi162 in reduce_add_blocks, LLVM will then generate vpmaddwd.

The binary generated using nightly rustc did not exhibit any inlining failures (confirmed using objdump). Using RUSTFLAGS="-Ctarget-cpu=x86-64-v3" also did not restore the benchmark results.

[folkertdev]

Lol. The code uses a dummy multiplication with an all-one vector to get vpmaddwd. With the new stdarch implementation, that multiplication is optimized out before vpmaddwd can be selected

https://godbolt.org/z/7fKnj9qe4

So, maybe we can get LLVM to optimize that differently, but for now reverting to use the LLVM intrinsics directly seems like the best option (especially given that this trick is used in a widely-used algorithm, e.g. the zlib-rs implementation runs into the same issue).

[apiraino]

Assigning priority (discussion on Zulip).

@rustbot label -I-prioritize +P-medium