Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A more resilient build chain for document collections #194

Open
ronaldtse opened this issue Dec 13, 2020 · 2 comments
Open

A more resilient build chain for document collections #194

ronaldtse opened this issue Dec 13, 2020 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@ronaldtse
Copy link
Contributor

This issue is to decide/develop a more resilient build child for people who work on document collections. I think this ties into Metanorma collection work @abunashir is working on (#180) and we could fix things from there.

From @Kritner below. This flow is also one that I know some others (and I) work with, so it is indicative of how people build collections.

Perhaps my current workflow context is important.

  • Work on document x locally from a clone of a fork of our main repository
  • Finish work on document x
  • Build document x locally
  • Ensure document meets expectations
  • Commit changes to document x and push up to fork of main repository
  • PR fork to main
  • Wait for successful build to merge PR (at a minimum, we may wait on the timing of other moving parts to merge as well)
  • After successful build, push all artifacts to github pages
    • this step is where "the problem lies", in that the "silent failure" documents were never produced, and GH pages now does not have these documents available to it. Though this may be an issue of force pushing the gh-pages branch, not super familiar with the GH actions.
  • If build fails, take no action on github pages

Given the above scenario, I only ever worked on document x, so I'm being careful about monitoring its progress. However, in this case documents 1...n failed to build in both our PR build as well as our "merge PR" build, without any sort of indication of said failure, and without having worked on any document in the range of 1...n.

It would be one thing if document x "silently failed" to build, as we know to be wary of that document as we're actively working on it, but that's not the case here, documents that weren't touched are now "silently failing" due to a change in a dependency down stream.

Perhaps a "strict" mode or similar would work, though we have plenty of warnings we'll need to work through.

My personal preference would be if any one document in the batch fails, even if 99% pass, the overall build is deemed a failure. This could be on our end as well in regards to how we're doing our make file, but I haven't really looked closely into how all the pieces fit together as it relates to the make file and the github action.

Originally posted by @Kritner in #189 (comment)

@ronaldtse ronaldtse added the enhancement New feature or request label Dec 13, 2020
@ronaldtse
Copy link
Contributor Author

We have just merged #180 which will aid this issue and gives us the possibility to propagate the error codes from individual document compilation.

As @Kritner mentioned, the problem is at this step:

  • After successful build, push all artifacts to github pages
    • this step is where "the problem lies", in that the "silent failure" documents were never produced, and GH pages now does not have these documents available to it. Though this may be an issue of force pushing the gh-pages branch, not super familiar with the GH actions.

My personal preference would be if any one document in the batch fails, even if 99% pass, the overall build is deemed a failure. This could be on our end as well in regards to how we're doing our make file, but I haven't really looked closely into how all the pieces fit together as it relates to the make file and the github action.

Now with the metanorma site generation command in place, it is possible to define different behavior:

  • --strict: once a failure occurs, bail out
  • --continue-on-error: don't bail out on error, at the end display errors but return failure code.

Output necessary:

  • A summary at the end should tell the user what documents have succeeded and what documents have failed.
  • Warnings should be made available in some manner at the end, a summary with per-document details prepended with the document's path.

And we can provide different error codes per situation:

  • 0: All documents compiled successful. A success message should be shown.
  • 1: Some failure occurred. The errored-out document should be warned out at the end.

This is not unlike the spec/test outputs, where a summary is provided (test successes) and the individual test errors are singled out for developer attention.

@abunashir can you help design this feature and proceed with it? Thanks!

@ronaldtse
Copy link
Contributor Author

We are facing this issue again:

When PDF generation fails, we MUST fail the whole build process at the end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants