Integrate Developer Experience Documentation/Scripting into Catalyst Forge

[stevenj]

Problem:

We currently use just and Justfiles to help document and automate tasks which developers do when working on the code bases.
This works fine, but has some limitations.
We would also like to reduce the number of minimum dependencies required to work on a Catalyst Project to:

1. Docker
2. Earthly
3. Catalyst Forge CLI

Solution:

Replace just with a command runner built inside Catalyst Forge.
We follow a principle outlined in: https://en.wikipedia.org/wiki/Literate_programming

Proposal.
In a Project we currently require two files:

• Earthfile
• blueprint.cue

To these we add a third optional file, which lives beside these files in the repo (for discoverability):

• Developer.md

Developer.md is both Documentation that developers need in order to be able to effectively and quickly work on the project, and embeds common tasks which developers do regularly.
In essence, this file is a regular markdown file, and code blocks can contain scripts which are executable, on demand by the forge cli.
This approach should improve our developer documentation and on-boarding.

We employ a certain structure to the markdown to allow documentation to be extracted, and for commands to be reliably found.

The Forge CLI will be enhanced using this library to parse the Developer.md files:
https://github.com/yuin/goldmark

Tasks

• Create initial subcommand to work with the markdown definition #65
• Add autocompletion when looking up commands from the markdown definition #66
• Implement markdown files discovery within the working repository #73
• Implement the markdown file to support target parameters.
• Implement the markdown file to support target platforms.
• Implement the markdown file to support denpendencies commands.
• Define proper documentation/specification

[stevenj]

Example Developer.md format for further discussion:

# My cool dev docs

## Run Some Command !!

''' sh
  echo "the fallback, if nothing else is defined"
'''

### Linux

''' python
  print("Cool")
'''

### Linux:arm64

''' sh
  echo "Because maybe python is broken??"
'''

### Windows

''' powershell
  echo "Does this even work in powershell??"
'''

### Mac:Arm64

''' sh
  echo "Because its always got to be different, even though its really just BSD Unix"
'''

This provides a run-some-command command, which is derived from Layer 2 headings.

Layer 3 headings define platform specific versions on the command.
The parser finds the first code block (sorted by platform) in a command that it can run .
When generating cli docs for a command, any platforms not supported locally are ommitted from the docs.

The language specifier can be checked if the code is runnable.

• sh = Run in the default system shell.
• bash = Literally run in bash, regardless of the shell the user was running.
• python = Run inside the system python.

If the language isn't a known language, it's not considered a command, just documentation.

Problems to solve:

• how do we indicate parameters that a command might take.
• Would it be useful to mark a command as runnable on multiple platforms. (For example, a command might run on any linux/mac, but windows needs a special version).
• Anything else.

[stevenj]

First Task:
Add a simple markdown cli command to forge that works like this:

forge devx <markdownfile.md> <command_name>

This parses the markdown file, finds the command by name, and executes anything inside a ''' sh ''' code block, immediately after the command name.

command-name comes from the 3rd level headings in the file.
Remove all whitespace from the front and back of the heading, and any special characters, and replace spaces with dashes. So:

### This is my  $$  555 command !!

becomes:

this-is-my-555-command