Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: nargo test -q (or nargo test --format terse) #6776

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

asterite
Copy link
Collaborator

@asterite asterite commented Dec 11, 2024

Descriptio

Made on top of #6728

Problem

Towards #6733

Summary

Eventually we'll have nargo test --format json, which is what #6733. As a first step I wanted to introduce the format flag and, to try that abstraction in code I introduced a terse formatter similar to Rust.

On failure we don't immediately print the error. Instead, it's printed at the end (like in Rust). I can imagine this could be used in CI to reduce the output verboseness, because I guess all that matters in CI is seeing which tests failed. NOTE: I already added this -q flag to our external checks, you can see how it looks, I think it's a bit better as it's less noisy.

nargo-test-q

Like in Rust, there are two ways to trigger this terse behavior: -q (or --quiet) or --format terse. We could just keep --format terse but I think -q is convenient (and format is more general as it will soon support json).

Here's the output when all tests pass:

image

Additional Context

  1. The proptest output that shows up in the middle of the run will be gone with fix: disable failure persistance in nargo test fuzzing #6777
  2. I just noticed that we have duplicate and now outdated code to run stdlib tests here. We've been able to run nargo test inside noir_stdlib for a while now so we can probably move those to a CI test matrix (in this PR I tried to reuse the code from nargo_cli but I couldn't because nargo_cli is a binary and we can't use that binary as a dependency in test integrations).

Documentation

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

Copy link
Contributor

github-actions bot commented Dec 11, 2024

Compilation Sample

Program Compilation Time %
sha256_regression 0m1.528s 2%
regression_4709 0m1.506s -1%
ram_blowup_regression 0m17.017s -1%
private-kernel-tail 0m1.496s 17%
private-kernel-reset 0m9.001s 0%
private-kernel-inner 0m2.724s 17%
parity-root 0m0.947s -17%
noir-contracts 2m39.114s -1%

Copy link
Contributor

github-actions bot commented Dec 11, 2024

Peak Memory Sample

Program Peak Memory %
keccak256 78.17M 0%
workspace 121.92M -1%
regression_4709 294.72M 0%
ram_blowup_regression 2.44G 0%
private-kernel-tail 208.76M 0%
private-kernel-reset 861.45M 0%
private-kernel-inner 307.68M 0%
parity-root 174.42M 0%

@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from a4d7df8 to 94aabde Compare December 11, 2024 19:59
Base automatically changed from ab/nargo-test-improvements to master December 12, 2024 02:10
@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from 94aabde to df37a64 Compare December 12, 2024 09:36
@asterite asterite added the run-external-checks Trigger CI job to run tests on external repos label Dec 12, 2024
@asterite asterite force-pushed the ab/nargo-test-machine-readable-output branch from fd0e854 to 04a53ea Compare December 12, 2024 10:23
@asterite asterite requested a review from a team December 12, 2024 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-external-checks Trigger CI job to run tests on external repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant