would like support for "cmd <source>... <destination>"

[tdterry]

I have a cp-like command that takes multiple sources and a single destination. I'm trying to replicate that with docopt, but it consumes all of the input args for the ... match and then fails to match . The parser either needs a way to put back input tokens and try again (back tracking) or it needs to consider the next token (look ahead).

I tool a look at the source, for match(). I couldn't see any way to make this change. If you have any suggestions, I would be happy to work on a patch.

[sourcefrog]

See also docopt/docopt#46 proposing to add this to the Python implementation, and apparently rejected.

To me the implementation there is a straightforward way to write it: a repeated item can optionally be followed by a fixed number of non-repeating items.

[keleshev]

docopt/docopt#46 was closed mostly because we were afraid of too much complexity in implementation and ambiguity in semantics. Right now the semantics is simple—matching is greedy.

If you make matching non-greedy you need to define some semantics that would resolve cases like:

cmd <arg1>... [<arg2>] <arg3>

cmd <arg1>... <arg2> <arg3>...

This starts to be a regular-expression-matching problem. But unlike in regular expressions, we don't have a way to control greediness as with ? operator (like a.*?).

If anyone can come up with nice simple solution to that, I would be more than happy to incorporate it in Python version of docopt.

[sourcefrog]

I respectfully disagree: you don't need to resolve those cases because
nobody wants to use them, because they are inherently ambiguous to a human
reader, in a way that cp is not.
On Wed Feb 05 2014 at 03:05:34, Vladimir Keleshev [email protected]
wrote:

docopt/docopt#46 docopt/docopt#46 was closed
mostly because we were afraid of too much complexity in implementation and
ambiguity in semantics. Right now the semantics is simple—matching is
greedy.

If you make matching non-greedy you need to define some semantics that
would resolve cases like:

cmd ... []

cmd ... ...

This starts to be a regular-expressions-matching problem. But unlike in
regular expressions, we don't have a way to control greediness as with ?operator (like
a.*?).

If anyone can come up with nice simple solution to that, I would be more
than happy to incorporate it in Python version of docopt.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/5#issuecomment-34074827
.

[keleshev]

If you don't want to allow these cases, then you have another problem—now you need to distinguish them. Should it be a whitelist, you think?

<arg1>... <arg>—allowed
<arg1>... [<arg>]—not allowed
[<arg1>] [<arg2>] <arg3>—hm, that doesn't seem ok?
[<arg1>] <arg2> <arg3>—hm, that seems ok?

[sourcefrog]

One possible rule is the one I gave above, restated to be more unambiguous as:

a command containing repeated or optionals item can optionally be terminated by a fixed number of non-repeating items.

Then these are OK and unambiguous:

<arg1>... <arg>
[<arg1>] [<arg2>] <arg3>
[<arg1>] <arg2> <arg3>

but this is not:

<a>... [<b>]

I think the last of them is contrived and not useful, nor very obvious to humans how it's supposed to be interpreted. I guess this would be a command that can either take

cmd a
cmd a b
cmd a1 a2 a3 b

ie you must take one a, then if possible one b, then the rest of them are a. Perhaps it is better spelled out as two alternatives, which is probably how I'd think of it:

cmd a
cmd a... b

[keleshev]

There is definitely ambiguity for me in [<a>] [<b>] <c>. If you pass 1 2 should it be <a>=1 <c>=2 or <b>=1 <c>=2? I can't say.

[sourcefrog]

You could reasonably say that the optional arguments fill up from the left,
as they do today if there is no trailing required arguments. This is the
same rule that makes [A] [B] unambiguous.

But, anyhow: if this is too ambiguous, it doesn't have to be supported. The
same reason it's hard to implement in docopt makes it unlikely to be used.

But there are lots of programs that have cp-like syntax, ie with one
trailing mandatory argument: cp, mv, git, bzr, hg, rsync, duplicity, scp,
gsutil, ... People (well at least me) would like to write more programs in
that style.

On Sun Feb 09 2014 at 04:23:02, Vladimir Keleshev [email protected]
wrote:

There is definitely ambiguity for me in [] [] . If you pass 1 2should it be =1
=2 or =1 =2. I can't say.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/5#issuecomment-34549702
.

[keleshev]

I would love to write programs in that style as well. We just need to come up with a rule that is easy to convey to the user. Something like: A pattern with single ... can be followed by one or more non-optional positional arguments. But that doesn't take into account all cases.

[badslug]

Rather than take into account all cases with a general solution, maybe the best place to start is to just handle the most common case: a pattern with a single ... followed by a single non-optional last argument which is always parsed with the semantics of cp: one or more source items, and one final target item. It seems the vast majority of tools we would ever write falls into that one case. Maybe even more specifically we could make it parse only the option when that single ... is named "source" and the final non-optional argument is named "target". It's a special case (which is not ideal) but the special case covers 90% of actual use-cases.

[smaftoul]

any news on this ?
I also need a source... target positional argument.
Is this blocked by the python implementation to stay compatible ?

[smaftoul]

@mboersma any reasons you labeled this wontfix ?
Is it to keep this implementation compatible with python's one ?

[mboersma]

@smaftoul I think I marked this wontfix because of @keleshev's arguments above, and because yes, it would need to be supported in Python docopt first since all the implementations should be broadly the same.

Having said that, that decision was years ago, and in the meantime we've seen docopt stagnate. I think the implementations work reasonably well as-is, but there has been little maintenance. It seems unlikely we could get any changes made to the core Python implementation, and I'd rather not have the go implementation diverge from that. But I'm just someone who volunteered to help maintain this library and if there's a PR with community support I would certainly review it.