shouldn't showing stack traces (a la --debug) be the default behaviour?

[aspiers]

Issue description

As mentioned in #219, it is possible to cause stestr to crash with an error min() arg is an empty sequence, but with no stacktrace. Adding --debug gives the stacktrace, but this begs the question - why are stack traces being hidden by default? It gives the false impression that a crash such as this one was the fault of the user, but as a result the error message will focus on something which went wrong internally, and as a result will generally appear cryptic to the user.

Steps to reproduce the problem

Exploit a bug such as the one mentioned in #219 in order to cause stestr to crash.

Expected behavior

The stacktrace is shown, making it obvious that this was a crash not a usage error caused by the end user.

Actual behavior

Only the error message is shown, with no indication the user hit a bug in stestr.

Specifications like the version of the project, operating system, or hardware

Same as in #219.

[mtreinish]

I agree with this, and always disliked the error message only without a stack trace by default. But, this comes from the default behavior in cliff and I do not know if there is a way to change how cliff behaves on this. If there is I'm definitely in favor of making sure we always print stack traces on unhandled exceptions

[aspiers]

@mtreinish commented on 29 Jan 2019, 17:43 GMT:

I agree with this, and always disliked the error message only without a stack trace by default. But, this comes from the default behavior in cliff and I do not know if there is a way to change how cliff behaves on this. If there is I'm definitely in favor of making sure we always print stack traces on unhandled exceptions

Well, as you well know cliff is Open Source maintained by excellent folks ;-) So I think we can confidently say there is a way to change how cliff behaves on this, even if that way requires some code changes :-)

Based on a quick glance it appears to have been added here:

openstack/cliff@276e8a4

so it's time to chat with @dhellmann - I'm sure he had a good reason for adding it :-) Maybe we can catch him in IRC.

[dhellmann]

Stack traces are a terrible way to report an error to a normal application user, so I don't think we want to change the default behavior in cliff. If you wanted to change the default for stestr, then we could update cliff to make it possible to override the default. It may already be possible to do that by modifying self.options.debug in the initialize_app() method, so give that a try first.

[aspiers]

@dhellmann commented on January 30, 2019 3:52 PM:

Stack traces are a terrible way to report an error to a normal application user

Yes, but as stated in IRC, I think a cryptic one-liner like min() arg is an empty sequence is even worse, because it leaves the user wondering whether they are supposed to understand that, and whether it was their fault. With a stack trace it's more obvious. Furthermore, if it's hard to reproduce, then re-running with --debug in order to capture essential debug to include in a bug report is not necessarily an option.

So here's an idea for how we could do even better:

1. Keep --debug behaviour as it is.

2. When --debug is not specified and an exception is caught, write the full error and stack trace to a file, and then output a much more user-friendly error to the terminal making it clear that this was a crash, e.g.

   BUG: min() arg is an empty sequence
   
   Unfortunately $PROGRAM unexpectedly crashed.  Please consider searching
   the issue tracker at
   
       $URL
   
   to see if this is a known issue, and if it hasn't already been reported,
   please report it, including the stack trace which has been saved to:
   
       $TRACE_FILE
   

Of course since this is handled in cliff, we'd want a way for application writers to customise the error message. @dhellmann Would you consider a patch to implement this?

[aspiers]

Also, specifying the URL of an issue tracker could be optional for the application author, in which case the default error template could automatically adjust depending on whether it's provided or not.

[aspiers]

I actually did something similar in another program of mine a long time ago, where there was even the possibility to guess that the crash was associated with known bugs:

https://github.com/aspiers/ly2video/blob/17327d4b021beb71a7e00d3995b79257eef330a0/ly2video/utils.py#L56