Modernize command-line parsing with optparse-applicative

[mitchellwrosen]

Is there any interest in replacing the System.Console.GetOpt code with a more modern optparse-applicative style options parser? If so, hm - maybe I'll take a stab. It could be a lot of work for little benefit, unless the existing options parsing code is slowing people down.

[hvr]

@mitchellwrosen well for one, there's issue of incurring an additional dependency in lib:Cabal which becomes a ghc boot package -- but we can workaround that by vendoring a subset of optparse-applicative into lib:Cabal

More importantly we'd need to figure out if all CLI features we currently have, and those which we intend to have in the future (such as e.g. context-sensitive tab-completion, whose competition candidates are retrieved lazily on-demand; e.g. consider cabal new-build exe:<tab>) can be expressed in optparse-applicative's framework.

[quasicomputational]

I've entertained a similar thought, but I shelved it because of the backwards-compatibility concerns.

We'd want a really, really thorough test-suite for command-line parsing to feel comfortable, I think, so that would be a first step to moving towards a different parsing library. (Besides all the usefulness of more tests catching regressions early and all that.)

[mitchellwrosen]

@hvr I'm not 100% positive optparse-applicative can parse everything exactly as Cabal does, but certain features (like "forwarding" arguments to subcommands so that cabal blah --flag is the same as cabal --flag blah when cabal itself does not have a global --flag) come for free. There's also a bash completion framework that I haven't tried to use, but it does allow IO.

@quasicomputational Good point, extensively testing the existing CLI would be the first step to refactoring it.

[mmhat]

Note that this would also fix #1257.

[michaelpj]

Would this really incur additional boot dependencies? AIUI GHC needs Cabal, but not cabal-install, and presumably CLI parsing should live in cabal-install?

[andreabedini]

The cli parsing is a mess. cabal-install's is bolted on top of Cabal's. They literally share the same types.
Perhaps this is one more reason to move cabal-install to optparse-applicative, so we separate the two clis.
I am afraid it's a fair bit of work though, at least in terms of lines of code.
Also note that v1 and v2 commands have different code paths for cli parsing. Which adds another level of complexity.

[Bodigrim]

FWIW optparse-applicative is Applicative: you can inspect Parser to build a QuickCheck-style generator of arbitrary arguments, accepted by it. This could be helpful to test that a new command-line parser behaves similarly to the old one.

Isn't it time to bury v1- commands? https://mail.haskell.org/pipermail/cabal-devel/2020-September/010488.html suggests so.

[ulysses4ever]

Isn't it time to bury v1- commands?

Cf. #9107 (comment)

[BurningWitness]

I want to extend on the idea I mentioned in passing on Reddit before (the crossed out part of my comment here).

There are roughly three ways to parse commandline options in Haskell as we know it:

• getopt. Cheap, fast and dirty, converts options into a list of [a]. The resulting list of options must be combined in some second parsing stage, which may also have errors of its own;

• optparse-applicative. A relatively complex modern library, converts options into boilerplate. Since it's Applicative, if any of your options overlap in meaning or effect, you will also need a second parsing stage;

• cmdargs#System.Console.CmdArgs.Explicit. Bloated abomination from 2010s. Uses a default state s and assigns each option to a Maybe String -> s -> Either Fail s function. The output can thus be the final state, so no second stage required.

cmdargs in my opinion is the closest to the correct answer, even if it's hard to see behind the pages upon pages of convenience functions and extensions. If distilled, both parsing and printing take up fewer than 500 lines and can be extended however needed. The template is inspectable by construction. For fun and performance indexing can be extended to use lazy radix trees, though strict containers are already fast enough for the job.

The end goal would be for each component to have a completely pure, completely separate Haskell datatype which it uses as an input. The resulting parser can sit in Distribution.Console.Options for the time being, but it could also become a boot library since it's general-purpose.

"forwarding" arguments to subcommands so that cabal blah --flag is the same as cabal --flag blah when cabal itself does not have a global --flag

If --flag requires an argument, then --flag blah means --flag=blah and rearranging it would change the meaning of the command. It can be partially worked around (since you would be able to check Cabal's option list), but I don't think it's good design.

It would make sense if --flag is an option shared by both cabal and its subcommand, but at that level it makes more sense for cabal to simply add that option as a part of the call, not as some generic option relay.

[BurningWitness]

After reading through System.Console.GetOpt it turned out s -> Either String s can be used as a type instead. It's still quirky (result could just be Either String (s, [String])) and slow (list lookups), but the spirit is there.