Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing HasCallStack to writeGoldenFile, reportGoldenFileMissing, checkAgainstGoldenFile #70

Merged
merged 1 commit into from
May 16, 2024

Conversation

carbolymer
Copy link
Contributor

This PR adds HasCallStack constraint to few functions doing the comparisons between strings or files. This allows for reporting where the failure (the execution of e.g. checkAgainstGoldenFile) occured. The downside is that the information message about the need to regenerate the files is shown at the failure location instead at the end of the error log. Sample failure after this change:

            149 ┃   -- gov-state
            150 ┃   do
            151 ┃     -- to stdout
            152 ┃     execCli' execConfig [ eraName, "query", "gov-state" ]
                ┃     │ ━━━━ command ━━━━
                ┃     │ /home/mgalazyn/workspace/iohk/dist-newstyle/build/x86_64-linux/ghc-9.8.2/cardano-cli-8.23.1.0/x/cardano-cli/build/cardano-cli/cardano-cli conway query gov-state
            153 ┃       >>=
            154 ┃         (`H.diffVsGoldenFile`
                ┃         │ Reading file: test/cardano-testnet-test/files/golden/queries/govStateOut.json
                ┃         │ Golden test failed against golden file.
                ┃         │ To recreate golden file, run with RECREATE_GOLDEN_FILES=1.
                ┃         ^^^^^^^^^^^^^^^^^^^^^
                ┃         │ 656c656
                ┃         │ <         "tag": "NoPParamsUpdate"
                ┃         │ ---
                ┃         │ >         "tag": ""
            155 ┃             "test/cardano-testnet-test/files/golden/queries/govStateOut.json")
            156 ┃     -- to a file
            157 ┃     let govStateOutFile = work </> "gov-state-out.json"
            158 ┃     H.noteM_ $ execCli' execConfig [ eraName, "query", "gov-state", "--out-file", govStateOutFile]
            159 ┃     H.diffFileVsGoldenFile
            160 ┃       govStateOutFile
            161 ┃       "test/cardano-testnet-test/files/golden/queries/govStateOut.json"
            162 ┃
            163 ┃   -- drep-state
            164 ┃   do
            165 ┃     -- to stdout
            166 ┃     -- TODO: deserialize to a Haskell value when
            167 ┃     -- https://github.com/IntersectMBO/cardano-cli/issues/606 is tackled
            168 ┃     dreps :: Aeson.Value <- H.noteShowM $ execCliStdoutToJson execConfig [ eraName, "query", "drep-state", "--all-dreps"]
            169 ┃     assertArrayOfSize dreps 3
            170 ┃     -- to a file
            171 ┃     let drepStateOutFile = work </> "drep-state-out.json"
            172 ┃     H.noteM_ $ execCli' execConfig [ eraName, "query", "drep-state", "--all-dreps"
            173 ┃                                    , "--out-file", drepStateOutFile]
            174 ┃     _ :: Aeson.Value <- H.readJsonFileOk drepStateOutFile
            175 ┃     pure ()
            176 ┃
            177 ┃   H.success

            This failure can be reproduced by running:
            > recheckAt (Seed 17230671044673320290 2354375817147678641) "1:" CliQueries

        Use "--pattern '$NF ~ /CliQueries/' --hedgehog-replay '1: Seed 17230671044673320290 2354375817147678641'" to reproduce from the command-line.

referenceLines <- List.lines <$> H.readFile goldenFile
let difference = getGroupedDiff actualLines referenceLines
case difference of
[] -> pure ()
[Both{}] -> pure ()
_ -> do
H.note_ $ unlines
[ "Golden test failed against golden file: " <> goldenFile
[ "Golden test failed against the golden file."
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path is logged already by H.readFile above.

@newhoggy newhoggy merged commit 93b7480 into main May 16, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants