chore(ci): Extend compiler memory report to external repos #6768

vezenovm · 2024-12-10T22:06:43Z

Description

Problem*

Followup to #6750

Summary*

This report essentially copies the process in #6750 for recording additional benchmarks from external repos. I made two new jobs external_repo_memory_report and upload_memory_report. I debating making the uploading and the uploading of the compilation report to be one job but I decided it would be better to keep them separate.

Additional Context

The memory report job for noir-contracts takes ~29 minutes with ~9G for peak memory. I commented it out for now due to the time it takes and was not sure we would want to be running this on every job.

I was thinking it may be best to simply generate benchmark reports for external repos only when specified by the label run-external-checks, similar to our testing of external repos.

Documentation*

Check one:

No documentation needed.
Documentation included in this PR.
[For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

I have tested the changes locally.
I have formatted the changes with Prettier and/or cargo fmt on default settings.

…into mv/compilation-report-external

github-actions · 2024-12-10T22:12:50Z

Compilation Sample

Program	Compilation Time
sha256_regression	0m1.528s
regression_4709	0m1.571s
ram_blowup_regression	0m17.353s
private-kernel-tail	0m1.511s
private-kernel-reset	0m9.492s
private-kernel-inner	0m2.587s
parity-root	0m0.919s
noir-contracts	2m53.443s

.github/workflows/reports.yml

github-actions · 2024-12-11T03:03:35Z

Peak Memory Sample

Program	Peak Memory
keccak256	81.38M
workspace	122.05M
regression_4709	333.59M
ram_blowup_regression	2.55G
private-kernel-tail	232.49M
private-kernel-reset	1.24G
private-kernel-inner	338.32M
parity-root	199.36M

…g/noir#6747) chore: Use `NumericType` not `Type` for casts and numeric constants (noir-lang/noir#6769) chore(ci): Extend compiler memory report to external repos (noir-lang/noir#6768) chore(ci): Handle external libraries in compilation timing report (noir-lang/noir#6750) feat(ssa): Implement missing brillig constraints SSA check (noir-lang/noir#6658) fix: Do not merge expressions that contain output witnesses (noir-lang/noir#6757) fix: parser would hand on function type with colon in it (noir-lang/noir#6764) chore(docs): Update branding (noir-lang/noir#6759) feat(cli): Run command on the package closest to the current directory (noir-lang/noir#6752) chore: lock CI to use ubuntu 22.04 (noir-lang/noir#6755) chore: free memory for silenced warnings early (noir-lang/noir#6748) chore(stdlib)!: Remove Schnorr (noir-lang/noir#6749) fix: Improve type error when indexing a variable of unknown type (noir-lang/noir#6744) fix: println("{{}}") was printing "{{}}" instead of "{}" (noir-lang/noir#6745) feat: `std::hint::black_box` function. (noir-lang/noir#6529) feat(ci): Initial compilation report on test_programs (noir-lang/noir#6731) chore: Cleanup unrolling pass (noir-lang/noir#6743) fix: allow empty loop headers (noir-lang/noir#6736) fix: map entry point indexes after all ssa passes (noir-lang/noir#6740) chore: Update url to 2.5.4 (noir-lang/noir#6741) feat: Order attribute execution by their source ordering (noir-lang/noir#6326) feat(test): Check that `nargo::ops::transform_program` is idempotent (noir-lang/noir#6694) feat: Sync from aztec-packages (noir-lang/noir#6730)

chore: Use `NumericType` not `Type` for casts and numeric constants (noir-lang/noir#6769) chore(ci): Extend compiler memory report to external repos (noir-lang/noir#6768) chore(ci): Handle external libraries in compilation timing report (noir-lang/noir#6750) feat(ssa): Implement missing brillig constraints SSA check (noir-lang/noir#6658) fix: Do not merge expressions that contain output witnesses (noir-lang/noir#6757) fix: parser would hand on function type with colon in it (noir-lang/noir#6764) chore(docs): Update branding (noir-lang/noir#6759) feat(cli): Run command on the package closest to the current directory (noir-lang/noir#6752) chore: lock CI to use ubuntu 22.04 (noir-lang/noir#6755) chore: free memory for silenced warnings early (noir-lang/noir#6748) chore(stdlib)!: Remove Schnorr (noir-lang/noir#6749) fix: Improve type error when indexing a variable of unknown type (noir-lang/noir#6744) fix: println("{{}}") was printing "{{}}" instead of "{}" (noir-lang/noir#6745) feat: `std::hint::black_box` function. (noir-lang/noir#6529) feat(ci): Initial compilation report on test_programs (noir-lang/noir#6731) chore: Cleanup unrolling pass (noir-lang/noir#6743) fix: allow empty loop headers (noir-lang/noir#6736) fix: map entry point indexes after all ssa passes (noir-lang/noir#6740) chore: Update url to 2.5.4 (noir-lang/noir#6741) feat: Order attribute execution by their source ordering (noir-lang/noir#6326) feat(test): Check that `nargo::ops::transform_program` is idempotent (noir-lang/noir#6694) feat: Sync from aztec-packages (noir-lang/noir#6730)

) chore: Try replace callstack with a linked list (noir-lang/noir#6747) chore: Use `NumericType` not `Type` for casts and numeric constants (noir-lang/noir#6769) chore(ci): Extend compiler memory report to external repos (noir-lang/noir#6768) chore(ci): Handle external libraries in compilation timing report (noir-lang/noir#6750) feat(ssa): Implement missing brillig constraints SSA check (noir-lang/noir#6658) fix: Do not merge expressions that contain output witnesses (noir-lang/noir#6757) fix: parser would hand on function type with colon in it (noir-lang/noir#6764) chore(docs): Update branding (noir-lang/noir#6759) feat(cli): Run command on the package closest to the current directory (noir-lang/noir#6752) chore: lock CI to use ubuntu 22.04 (noir-lang/noir#6755) chore: free memory for silenced warnings early (noir-lang/noir#6748) chore(stdlib)!: Remove Schnorr (noir-lang/noir#6749) fix: Improve type error when indexing a variable of unknown type (noir-lang/noir#6744) fix: println("{{}}") was printing "{{}}" instead of "{}" (noir-lang/noir#6745) feat: `std::hint::black_box` function. (noir-lang/noir#6529) feat(ci): Initial compilation report on test_programs (noir-lang/noir#6731) chore: Cleanup unrolling pass (noir-lang/noir#6743) fix: allow empty loop headers (noir-lang/noir#6736) fix: map entry point indexes after all ssa passes (noir-lang/noir#6740) chore: Update url to 2.5.4 (noir-lang/noir#6741) feat: Order attribute execution by their source ordering (noir-lang/noir#6326) feat(test): Check that `nargo::ops::transform_program` is idempotent (noir-lang/noir#6694) feat: Sync from aztec-packages (noir-lang/noir#6730)

chore: Try replace callstack with a linked list (noir-lang/noir#6747) chore: Use `NumericType` not `Type` for casts and numeric constants (noir-lang/noir#6769) chore(ci): Extend compiler memory report to external repos (noir-lang/noir#6768) chore(ci): Handle external libraries in compilation timing report (noir-lang/noir#6750) feat(ssa): Implement missing brillig constraints SSA check (noir-lang/noir#6658) fix: Do not merge expressions that contain output witnesses (noir-lang/noir#6757) fix: parser would hand on function type with colon in it (noir-lang/noir#6764) chore(docs): Update branding (noir-lang/noir#6759) feat(cli): Run command on the package closest to the current directory (noir-lang/noir#6752) chore: lock CI to use ubuntu 22.04 (noir-lang/noir#6755) chore: free memory for silenced warnings early (noir-lang/noir#6748) chore(stdlib)!: Remove Schnorr (noir-lang/noir#6749) fix: Improve type error when indexing a variable of unknown type (noir-lang/noir#6744) fix: println("{{}}") was printing "{{}}" instead of "{}" (noir-lang/noir#6745) feat: `std::hint::black_box` function. (noir-lang/noir#6529) feat(ci): Initial compilation report on test_programs (noir-lang/noir#6731) chore: Cleanup unrolling pass (noir-lang/noir#6743) fix: allow empty loop headers (noir-lang/noir#6736) fix: map entry point indexes after all ssa passes (noir-lang/noir#6740) chore: Update url to 2.5.4 (noir-lang/noir#6741) feat: Order attribute execution by their source ordering (noir-lang/noir#6326) feat(test): Check that `nargo::ops::transform_program` is idempotent (noir-lang/noir#6694) feat: Sync from aztec-packages (noir-lang/noir#6730)

vezenovm added 30 commits December 9, 2024 22:30

update compilation_report script and job to handle external libraries

3338487

bring back accidental deleteion

a1104a0

missing job

d127872

remove job

aa82e83

Merge branch 'master' into mv/compilation-report-external

491bf47

write perms

4eba04b

Merge remote-tracking branch 'origin/mv/compilation-report-external' …

bc4f40f

…into mv/compilation-report-external

tests report w/o matrix

4a4ea20

remove dep on old job

c8e8444

test if reports runs

27d4d61

test w override

ce3f652

bring back upload_compilation_report

647f60e

just upload report

2fcfc9a

comment out upload

ed0c12f

bring back upload artifact

288c8d1

add debug

e678221

remove artifact upload

4ca7986

try bringing back again

537e2c7

do not specify working directory

b6f2179

bring back upload report

290d81c

change path

0f1cc54

specify dir path not file name

6aefe80

do not specify path

dee5f14

nit with brackets

308a2ef

try to trigger report

81910d2

test next job

804f231

update name

69cb0c9

remove the download

4e01594

use checout and download without path

e778e86

remove the download

25d39f8

vezenovm added 2 commits December 10, 2024 22:28

fix artifact name, some job names, and increase timeout

e17d18e

bump timeout

3d467f2

vezenovm commented Dec 11, 2024

View reviewed changes

.github/workflows/reports.yml Outdated Show resolved Hide resolved

vezenovm added 3 commits December 10, 2024 21:31

Update .github/workflows/reports.yml

5e557dc

move bench report merging to a script

349049c

update index

836c277

vezenovm added 6 commits December 11, 2024 03:06

never passed name arg

cc2c110

upload script location

00d732b

fix name

6568738

debug vars

4e16ae3

distinguish plural name

ca3cfa2

move compilation report merging to script and cleanup

5a4aa5f

vezenovm marked this pull request as ready for review December 11, 2024 04:51

fix comments

aced17d

Base automatically changed from mv/compilation-report-external to master December 11, 2024 14:43

TomAFrench and others added 2 commits December 11, 2024 15:20

Merge branch 'master' into mv/mem-report-external-repos

596b3a0

.

4cfbbb3

TomAFrench approved these changes Dec 11, 2024

View reviewed changes

TomAFrench enabled auto-merge December 11, 2024 15:44

TomAFrench added this pull request to the merge queue Dec 11, 2024

Merged via the queue into master with commit 2a840ca Dec 11, 2024
62 checks passed

TomAFrench deleted the mv/mem-report-external-repos branch December 11, 2024 16:05

AztecBot mentioned this pull request Dec 11, 2024

feat: Sync from noir AztecProtocol/aztec-packages#10483

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(ci): Extend compiler memory report to external repos #6768

chore(ci): Extend compiler memory report to external repos #6768

vezenovm commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 11, 2024 •

edited

Loading

chore(ci): Extend compiler memory report to external repos #6768

chore(ci): Extend compiler memory report to external repos #6768

Conversation

vezenovm commented Dec 10, 2024 • edited Loading

Description

Problem*

Summary*

Additional Context

Documentation*

PR Checklist*

github-actions bot commented Dec 10, 2024 • edited Loading

Compilation Sample

github-actions bot commented Dec 11, 2024 • edited Loading

Peak Memory Sample

vezenovm commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 11, 2024 •

edited

Loading