Add dynamic matrix generation for winpcap / npcap backends. #1950

Dimi1010 · 2025-09-07T16:51:40Z

This PR adds a new step to CI pipeline that dynamically generates the windows pcap backends matrix axis that are tested against.

Winpcap: Always available as a tested backend.
Npcap: Included as a backend if OEM credentials are provided to the workflow via repository secrets NPCAP_USERNAME and NPCAP_PASSWORD. These secrets are required for the silent installer to install the backend.

codecov · 2025-09-07T18:25:36Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.52%. Comparing base (92dc742) to head (f94090d).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files

@@           Coverage Diff           @@
##              dev    #1950   +/-   ##
=======================================
  Coverage   83.52%   83.52%           
=======================================
  Files         310      310           
  Lines       54905    54905           
  Branches    12010    12000   -10     
=======================================
  Hits        45857    45857           
+ Misses       7819     7798   -21     
- Partials     1229     1250   +21

Flag	Coverage Δ
alpine320	`75.97% <ø> (ø)`
fedora42	`76.09% <ø> (-0.03%)`	⬇️
macos-13	`81.66% <ø> (ø)`
macos-14	`81.66% <ø> (ø)`
macos-15	`81.66% <ø> (ø)`
mingw32	`70.37% <ø> (+0.01%)`	⬆️
mingw64	`70.33% <ø> (-0.03%)`	⬇️
rhel94	`75.82% <ø> (+<0.01%)`	⬆️
ubuntu2004	`60.25% <ø> (ø)`
ubuntu2004-zstd	`60.38% <ø> (+0.02%)`	⬆️
ubuntu2204	`75.77% <ø> (+0.01%)`	⬆️
ubuntu2204-icpx	`60.79% <ø> (ø)`
ubuntu2404	`76.00% <ø> (ø)`
ubuntu2404-arm64	`75.62% <ø> (ø)`
unittest	`83.52% <ø> (ø)`
windows-2022	`85.52% <ø> (ø)`
windows-2025	`85.55% <ø> (ø)`
winpcap	`85.55% <ø> (ø)`
xdp	`53.50% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

seladb · 2025-09-08T01:16:30Z

.github/workflows/build_and_test.yml

This is a great solution! However, it will add additional checks for PRs opened on the main repo vs. PRs opened in forks.
Ideally, I'd like the same number of checks, just use WinPcap instead of Npcap if not available

Why the same number of checks?

If npcap isn't available, doing a CI run with it marked as winpcap is redundant as you are running the winpcap job twice in that case?

The way it works now is that we have a combination of x64 / x86 and debug / release configurations and we attach some of them to WinPcap and others to Npcap. The assumption is that for most PRs it doesn't matter, but it still good to run with both options. I don't think we should have more jobs running on the main repo, but rather replace Npap jobs with WinPcap jobs on forked repo

We don't actually build release configs in the build_and_test CI as far as I know. Only debug.

It can be done so it uses Npcap backend if available and Winpcap if not. But that would have to be full replacement as it would be done in the matrix axis generation job, which would affect all Windows jobs.

I am worried as that would mean fully dropping Winpcap CI coverage on the main repo completely. My assumption is that most PRs from forked repos that want to contribute will target the main repo directly, so only main repo CI will run.

We don't actually build release configs in the build_and_test CI as far as I know. Only debug.

You're right, we don't have release builds, we should probably have at least one...

It can be done so it uses Npcap backend if available and Winpcap if not. But that would have to be full replacement as it would be done in the matrix axis generation job, which would affect all Windows jobs.

I don't think we should always use Npcap if available or WinPcap if not. Ideally we should have a mix of both, but in forked repos, fallback to WinPcap. For example:

visual-studio: strategy: matrix: include: - os: windows-2025 platform: "Visual Studio 17 2022" arch: Win32 pcap_lib: "winpcap" - os: windows-2025 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" # run on WinPcap in forked repos - os: windows-2022 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" # run on WinPcap in forked repos

In both main and forked repos we should have these 3 jobs, but in forked repos they should all use WinPcap

It can be done so it uses Npcap backend if available and Winpcap if not. But that would have to be full replacement as it would be done in the matrix axis generation job, which would affect all Windows jobs.

I don't think we should always use Npcap if available or WinPcap if not. Ideally, we should have a mix of both, but in forked repos, fallback to WinPcap. For example:

visual-studio: strategy: matrix: include: - os: windows-2025 platform: "Visual Studio 17 2022" arch: Win32 pcap_lib: "winpcap" - os: windows-2025 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" # run on WinPcap in forked repos - os: windows-2022 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" # run on WinPcap in forked repos

In both main and forked repos we should have these 3 jobs, but in forked repos they should all use WinPcap

~~Well, that can't be done with the dynamic generation since that generates an axis. It can't generate individual jobs.~~ See edit.

IMO, having the extra Npcap jobs is preferable than having a CI job that says that it is an npcap job, but surprise, it actually isn't. That is from both coverage and maintenance perspectives. Debugging a failed CI job that silently replaces a dependency with something else would be much harder than just not having the run job in the first place because the dependency is missing.

The way the jobs get named, people will see "pcap_lib=npcap" in the name and, without looking further in the code, will assume it uses npcap.

The jobs already cancel when a new push is done on a PR, so even if the CI is somewhat longer, it will abort if it is unnecessary.

Edit:
Upon further thought, I suppose you can set the pcap_lib to read from a variable for the flip flop jobs, but I kinda dislike that because of variable coverage. IMO, the main repo should not have less CI coverage than the forked repos. :/

@seladb Managed to get it to work. Can you check the VS job?

Maybe we can have something like this?

visual-studio: strategy: matrix: include: - os: windows-2025 platform: "Visual Studio 17 2022" arch: Win32 pcap_lib: "winpcap" run_on: all-repos - os: windows-2025 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" run_on: main-repo - os: windows-2025 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "winpcap" run_on: forked-repos - os: windows-2022 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "npcap" run_on: main-repo - os: windows-2022 platform: "Visual Studio 17 2022" arch: "x64" pcap_lib: "winpcap" run_on: forked-repos

It's not ideal to have so many jobs, but maybe it's the most explicit option, and also not too hard to implement

The most recent commit already does that.
The variable npcap-or-default resolves to 'npcap' if OEM credentials are available and 'winpcap' if not.

IMO, that is enough. Forked repos should be able to run the CI with Npcap with no changes, provided they supply their own OEM secret.

… Npcap is not available.

- Use Npcap backend for x64 jobs if available.

…as a json object.

seladb · 2025-09-11T07:08:21Z

.github/workflows/build_and_test.yml

+        pcap_lib: ${{ fromJSON(needs.define-windows-pcap-backend-matrix.outputs.pcap-backends) }}
+        sys: [mingw32, mingw64]
+        include: # Extends the matrix to add the env variable for each matching sys
+          - sys: mingw32
+            env: i686
+          - sys: mingw64
+            env: x86_64


Maybe we should define something similar to what we have in VS?

matrix: include: - sys: mingw32 env: i686 pcap_lib: "winpcap" - sys: mingw64 env: x86_64 pcap_lib: ${{ needs.define-windows-pcap-backend-matrix.outputs.npcap-or-default }}

If we do that the pcap-backends output is not needed

Updated 28a25f4 & 16aa358

The name of the job is a bit inaccurate now, but it can probably do.

seladb · 2025-09-12T05:50:19Z

Thank you for working on this @Dimi1010 ! 🙏

Add debug jobs for selecting winpcap or npcap matrix.

46f00b5

Dimi1010 added the CI label Sep 7, 2025

Dimi1010 added 10 commits September 7, 2025 20:12

Update step so it does not use secrets directly in bash.

6208d06

Fix syntax.

46fae38

Fix condition.

e69287d

Attempt to fix again.

226081c

Fix quotes.

d2dbca5

Fix expression again...

3a84024

temp remove the secret check

e513745

Fix case?

59ca967

Fix step outputs.

fe11a71

Test dynamic matrix on mingw job.

a7c2574

Dimi1010 added 7 commits September 7, 2025 21:38

Expand matrices on windows and mingw.

8120d04

Rename matrix generation job.

404c334

Add step to enable Npcap backend if OEM credentials are available.

bf056a5

Fix conditional.

6526248

Updated step desc.

4455c5a

Improve comments.

a725c85

Lint

922f531

Dimi1010 changed the title ~~Test dynamic matrix generation~~ Add dynamic matrix generation for winpcap / npcap backends. Sep 7, 2025

Dimi1010 added the Windows label Sep 7, 2025

Dimi1010 marked this pull request as ready for review September 7, 2025 19:41

Dimi1010 requested a review from seladb as a code owner September 7, 2025 19:41

Dimi1010 requested review from clementperon, tigercosmos and egecetin September 7, 2025 19:41

Dimi1010 mentioned this pull request Sep 7, 2025

[DRAFT] Add back Npcap when running in the main repo #1945

Closed

Dimi1010 added 2 commits September 7, 2025 22:52

Fix case.

99551bd

Add step explaining why NPcap is unavailable.

993ee6c

Fix quotes.

666f198

seladb reviewed Sep 8, 2025

View reviewed changes

Dimi1010 added 3 commits September 9, 2025 19:23

Add another output that prefers Npcap backend but will use Winpcap if…

6144ddc

… Npcap is not available.

Rollback windows jobs to a partial matrix.

0de92ff

- Use Npcap backend for x64 jobs if available.

Removed quotation marks from npcap-or-default as it will not be read …

01f76c1

…as a json object.

seladb reviewed Sep 11, 2025

View reviewed changes

Dimi1010 added 3 commits September 11, 2025 20:24

Rollback mingw jobs to single jobs.

28a25f4

Remove pcap-backends output.

16aa358

Merge branch 'dev' into ci/dynamic-pcap-backend

f94090d

seladb approved these changes Sep 12, 2025

View reviewed changes

Dimi1010 merged commit d5094c8 into seladb:dev Sep 12, 2025
43 checks passed

Dimi1010 deleted the ci/dynamic-pcap-backend branch September 12, 2025 06:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add dynamic matrix generation for winpcap / npcap backends. #1950

Add dynamic matrix generation for winpcap / npcap backends. #1950

Uh oh!

Dimi1010 commented Sep 7, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 7, 2025 •

edited

Loading

Uh oh!

seladb Sep 8, 2025

Uh oh!

Dimi1010 Sep 8, 2025 •

edited

Loading

Uh oh!

seladb Sep 8, 2025 •

edited

Loading

Uh oh!

Dimi1010 Sep 8, 2025 •

edited

Loading

Uh oh!

seladb Sep 8, 2025

Uh oh!

Dimi1010 Sep 8, 2025 •

edited

Loading

Uh oh!

Dimi1010 Sep 10, 2025

Uh oh!

seladb Sep 11, 2025 •

edited

Loading

Uh oh!

Dimi1010 Sep 11, 2025 •

edited

Loading

Uh oh!

seladb Sep 11, 2025

Uh oh!

Dimi1010 Sep 11, 2025

Uh oh!

seladb commented Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

Add dynamic matrix generation for winpcap / npcap backends. #1950

Add dynamic matrix generation for winpcap / npcap backends. #1950

Uh oh!

Conversation

Dimi1010 commented Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

seladb Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seladb Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seladb Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

seladb Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seladb Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Dimi1010 Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

seladb commented Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

Dimi1010 commented Sep 7, 2025 •

edited

Loading

codecov bot commented Sep 7, 2025 •

edited

Loading

Dimi1010 Sep 8, 2025 •

edited

Loading

seladb Sep 8, 2025 •

edited

Loading

Dimi1010 Sep 8, 2025 •

edited

Loading

Dimi1010 Sep 8, 2025 •

edited

Loading

seladb Sep 11, 2025 •

edited

Loading

Dimi1010 Sep 11, 2025 •

edited

Loading