Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add discard functionality for conditional tests #40

Merged
merged 21 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
05363bd
Add preliminary function prototype
BowTiedRadone Nov 1, 2024
3c72d66
Use the `preliminary function` for the test function early return
BowTiedRadone Nov 1, 2024
9413b6b
Retrieve property preliminary functions
BowTiedRadone Nov 1, 2024
88faca2
Pair test functions with preliminary functions
BowTiedRadone Nov 1, 2024
902442e
Add ultimate check for preliminary functions
BowTiedRadone Nov 1, 2024
f5858b2
Run preliminary function before the test function
BowTiedRadone Nov 1, 2024
7bbde3a
Adjust test output logs
BowTiedRadone Nov 1, 2024
2c1f32b
Conditionally run properties based on preliminary functions' response
BowTiedRadone Nov 2, 2024
cf248e7
Apply suggestions from code review
BowTiedRadone Nov 4, 2024
af4c8c9
Add new line in reporter logs
BowTiedRadone Nov 4, 2024
fbf8286
Tweak logging ansicolors
BowTiedRadone Nov 4, 2024
756bdd9
Remove unnecessary line
BowTiedRadone Nov 5, 2024
cb3f927
Rename `preliminary` to `discard` across the app
BowTiedRadone Nov 5, 2024
d154005
Refactor discard prototype comment
BowTiedRadone Nov 6, 2024
f0d8daf
Refactor test-discard pairing comment
BowTiedRadone Nov 6, 2024
ada9bc7
Refactor discard rules verification comment
BowTiedRadone Nov 6, 2024
3d12d0b
Refactor discard functions validation control flow
BowTiedRadone Nov 6, 2024
96130cf
Refactor control flow for test discarding
BowTiedRadone Nov 6, 2024
7eabfaf
Add property-based tests for discard utility functions
BowTiedRadone Nov 7, 2024
a1d8c70
Correct grammar in comments
moodmosaic Nov 8, 2024
34060ef
Remove extra spaces in log messages and misplaced `green` function call
moodmosaic Nov 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions example/contracts/slice.tests.clar
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@

(define-public (test-slice-list-int (seq (list 127 int)) (skip int) (n int))
(if
;; Early return if the input is invalid.
(or
(not (and (<= 0 n) (<= n 127)))
(not (and (<= 0 skip) (<= skip 127))))
;; Early return if the preliminary function returns false.
(not (can-test-slice-list-int seq skip n))
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
(ok true)
(let (
(result (contract-call? .slice slice seq skip n))
Expand All @@ -32,6 +30,28 @@
(asserts! (is-eq (len result) (to-uint n)) ERR_ASSERTION_FAILED_3)))
(ok true))))

BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
;; If a property test is conditional (https://www.cse.chalmers.se/~rjmh/
;; QuickCheck/manual_body.html#6), it requires a preliminary function to check
;; the input validity before running the test.
;;
;; The preliminary function must follow these rules:
;; - It must have read-only access.
;; - Its name should match the property test function's name, prefixed with
;; "can-".
;; - Its parameters should mirror those of the property test function.
;; - It must return a boolean indicating whether the inputs are valid.
;;
;; Rendezvous will first call the preliminary function; if it returns false, a
;; warning will be logged. In the future, with config file support, users will
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
;; be able to customize generated argument values based on the preliminary
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
;; function's result.
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
(define-read-only (can-test-slice-list-int (seq (list 127 int))
(skip int)
(n int))
(and
(and (<= 0 n) (<= n 127))
(and (<= 0 skip) (<= skip 127))))

(define-public (test-slice-list-uint (seq (list 127 uint)) (skip int) (n int))
(if
;; Early return if the input is invalid.
Expand Down
32 changes: 32 additions & 0 deletions property.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,38 @@ export const checkProperties = (
`\nStarting property testing type for the ${sutContractName} contract...`
);

// Search for preliminary functions, for each test function. This map will
// be used to pair the test functions with their corresponding preliminary
// functions.
const testContractsPreliminaryFunctions = new Map(
Array.from(testContractsAllFunctions, ([contractId, functions]) => [
contractId,
functions.filter(
(f) => f.access === "read_only" && f.name.startsWith("can-")
),
])
);

// Pair each test function with its corresponding preliminary function. When
// a test function is selected, Rendezvous will first call its preliminary
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
// function, if available, to validate that the generated test arguments are
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
// meaningful. This way, we are reducing the risk of false positives in test
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
// results.
BowTiedRadone marked this conversation as resolved.
Show resolved Hide resolved
const testContractsPairedFunctions = new Map(
moodmosaic marked this conversation as resolved.
Show resolved Hide resolved
Array.from(testContractsTestFunctions, ([contractId, functions]) => [
contractId,
new Map(
functions.map((f) => {
const preliminaryFunction = testContractsPreliminaryFunctions
.get(contractId)
?.find((pf) => pf.name === `can-${f.name}`);

return [f.name, preliminaryFunction?.name];
})
),
])
);

const radioReporter = (runDetails: any) => {
reporter(runDetails, radio, "test");
};
Expand Down