Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Foundry Mutation Testing #6588

Draft
wants to merge 74 commits into
base: master
Choose a base branch
from
Draft

Foundry Mutation Testing #6588

wants to merge 74 commits into from

Conversation

samparsky
Copy link

@samparsky samparsky commented Dec 12, 2023 •
edited
Loading

Motivation

This implements a simple mutation testing based on the gambit crate.

Checkout the issue to find out progress #478

Solution

It aims to implement simple operator-based mutation tests using the gambit crate.

Screenshot 2023-12-09 at 16 18 49

@samparsky
feat: add gambit crate
30d5bd2
@samparsky
chore: add dependencies for gambit
6777cc9
@samparsky
chore: add MutatorConfigBuilder, Mutator
8ce0351
@samparsky
chore: add FunctionFilter trait to common crate
afac7c0
@samparsky
chore: remove rayon unused dependency
e0ef02d
@samparsky
chore: rename gambit crate to mutator
1f6e1f3
@samparsky
chore: fix clippy comments
7ea5449
@samparsky
chore: remove comments
762ed6c
@samparsky
chore: add mutator artifact filter
129fd9f
@samparsky
chore: add artifact filter get_artifact_functions
fcff7e9
@samparsky
chore: update mutator config builder, add src_root
c32fd34
@samparsky
chore: apply rustfmt
556b0ba
@samparsky
chore: add mutate folder to forge/bin/cmd
c0c3011
@samparsky
chore: add filter.rs to mutate bin
9144d35
@samparsky
chore: add mutate commad filter args
1e6ce77
@samparsky
chore: add mutate filter args implementation
89b4e9e
@samparsky
chore: add mutate config to config crate
a3392bb
@samparsky
chore: add mutate to config struct
55036c8
@samparsky
test: fix cli/config mutate property
443e5e2
@samparsky
chore: add mutate test args struct
bf7d9cc
@samparsky
chore: add foundry_evm_mutator dep
9e43b70
@samparsky
chore: add mutate test args implementation
a2b491e
@samparsky
chore: make test(..) public; fix test
1093c64
@samparsky
chore: add mutate test summary struct and definitions
d59c2bb
@samparsky
chore: fix run_mutate
0d34b69
@samparsky
chore: add export flag to config
c9c9117
@samparsky
chore: unecessary println
bdaa5cf
@samparsky
chore: add mutate spinner reporter, test mode flag
fb61e15
@samparsky
chore: add test filter params to mutate config
78830e2
@samparsky
chore: add test filter params to mutate filter; add ContractFilter trait
532965a
@samparsky
chore: chunk and parallized compile stage
f513b4a
@samparsky
chore: ignore test file paths
05bb1f6
@samparsky
chore: fix comments
bf0edb4
@samparsky
chore: fix cargo comments
9c40a8f
@samparsky
chore: add solc jobs
94b78ec
@samparsky
fix(git): merge conflicts
0a84183
@samparsky
fix: merge conflicts, update traits
1089c52
@samparsky
fix: test checks; filters
cb74a7e
@samparsky
chore: fix mutatefilterargs debug struct name
f179e94
@samparsky
chore: add invariant runs check if > 0
0bbcfb4
@samparsky
chore: add fuzz runs check if > 0
3980d42
@samparsky
chore: make default_out_dir var
366898a
@samparsky
chore: add timeout mutation test result; html exporter
e323d01
@samparsky
chore: add progressreporter and rewrite tests
b6720c1
@samparsky
chore: change try_join_all to join_all
d450b0b
@samparsky
mend
ecba4c7
@samparsky
chore: add timeout, parallel, max_timeout_test config params
452f20e
@samparsky
chore: a couple of changes
3d3444a
@samparsky
chore: refactor parallel, filter
45bc838
@samparsky
chore: remove filter.rs
dd4117e
@samparsky
chore: add fuzz runs and seed to config
eae28bf
@samparsky
chore: add fuzz runs and seed to config for mutant test
4eb1aa5
@samparsky
chore: add mutant fuzz config
d34745c
@samparsky
chore: add get_contract_name to mutator crate
d74ef9e
@mds1 mds1 mentioned this pull request Apr 12, 2024
Merged
@zerosnacks zerosnacks modified the milestone: v1.0.0 Jul 31, 2024
@zerosnacks zerosnacks added the T-proof-of-concept Type: proof of concept label Jul 31, 2024
@zerosnacks
Copy link
Member

zerosnacks commented Dec 20, 2024 •
edited
Loading

Hi @samparsky,

Huge thanks for your PR! This is definitely a feature we will want to offer and I think Solar's frontend may have unlocked a step towards implementing mutation testing natively in Foundry in an elegant way.

Is this something you would be interested in exploring further / collaborate on?

cc @DaniPopes / @grandizzy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DaniPopes DaniPopes Awaiting requested review from DaniPopes DaniPopes will be requested when the pull request is marked ready for review DaniPopes is a code owner

@klkvr klkvr Awaiting requested review from klkvr klkvr will be requested when the pull request is marked ready for review klkvr is a code owner

@mattsse mattsse Awaiting requested review from mattsse mattsse will be requested when the pull request is marked ready for review mattsse is a code owner

@grandizzy grandizzy Awaiting requested review from grandizzy grandizzy will be requested when the pull request is marked ready for review grandizzy is a code owner

@yash-atreya yash-atreya Awaiting requested review from yash-atreya yash-atreya will be requested when the pull request is marked ready for review yash-atreya is a code owner

@zerosnacks zerosnacks Awaiting requested review from zerosnacks zerosnacks will be requested when the pull request is marked ready for review zerosnacks is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
T-proof-of-concept Type: proof of concept
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@samparsky @zerosnacks