Exception group stage4 - implement except*

[iritkatriel]

Example:

eg = ExceptionGroup('my eg', ValueError(12), ExceptionGroup('nested', TypeError(13), ValueError(42), KeyError(14)))

try:
    raise eg
except* OSError as e:
    print('caught OSError:', repr(e))
except* ValueError as e:
    print('caught ValueError:', repr(e))
except* KeyError as e:
    print('caught KeyError:', repr(e))
except* SyntaxError as e:
    print('caught SyntaxError:', repr(e))

Gives this output:

caught ValueError: ExceptionGroup('my eg', ValueError(12), ExceptionGroup('nested', ValueError(42)))
caught KeyError: ExceptionGroup('my eg', ExceptionGroup('nested', KeyError(14)))
Traceback (most recent call last):
  File "C:\Users\User\src\cpython\tmp.py", line 5, in <module>
    raise eg
ExceptionGroup: my eg
This exception has 1 sub-exceptions:
 ----------------- 1/1 -----------------
 ExceptionGroup: nested
 This exception has 1 sub-exceptions:
  ----------------- 1.1/1 -----------------
  TypeError: 13
  ----------------- end of 1.1 -----------------
 ----------------- end of 1 -----------------

TODO list:
(1) support raise inside the handler, and chaining of exception groups.
(2) nice syntax error when someone mixes except and except*.
(3) in ceval the JUMP_IF_NOT_EG_MATCH code peeks into the stack to see the val of the exception. Should this be DUPPED by the compiler like exc? (I'm not sure why the compiler is doing DUP_TOP just so that the JUMP_IF_NOT_EXC_MATCH code can pop it).
(4) rename exc --> nested_exceptions (or something like that). Make it immutable.
(5) rename ExceptionGroup.project() --> ExceptionGroup.split().

[gvanrossum]

Here are a few random comments. This PR is very big and I have not found the courage yet to try and understand every part of it, but the parts that I did check (in particular the parser and some of the tests) all check out.

[gvanrossum]

This reminds me -- is the recommended style except *E, or is it except* E?

[iritkatriel]

That's a good question, we started with except *E, but I found myself gravitating towards except* E because it feels like the * is operating on the except and not on the E. I find except* pretty, but then I wonder if this will be confusing:

>>> try:
...    raise ValueError
... except (*TypeError, *ValueError):
...    pass
...
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
TypeError: Value after * must be an iterable, not type
>>>

Well - "except *ValueError" doesn't care that * is operating on a non-iterable.

Maybe except* is really a new token (should we think again about catch?).

[1st1]

my 2c: I like except *E better because it resonates with argument unpacking syntax. I think that symmetry is what makes this proposal attractive in the first place.

[1st1]

Also, this is subtle and subjective, but except * doesn't look like a whole new syntax to me -- it's merely about modifying the existing construct to support unpacking. Whereas except* does feel like a new syntax, which perhaps will raise the bar of accepting this PEP a tad.

[iritkatriel]

I have two concerns with where we are at the moment. One is what I said above about naming except*- is the unpacking analogy valid? The syntax makes it look like we're unpacking the type or type tuple.

The second issue is about semantics: We add except* because we can't change except, but during the sprint we had the idea of adding catch with a view to it possibly being a replacement for except. Let's imagine we do this and rename except* to catch, and then users who write new code can just use catch and forget about except. Then we need to not wrap plain exceptions by an exception group, so

try:
   raise ValueError(12)
catch ValueError as e:
   """e is ValueError(12), not ExceptionGroup("", ValueError(12))"""

Con: the runtime type of e is less certain. Pro: You can use catch for all your exception handling needs (so you don't need to learn about except if you're writing new code).

[gvanrossum]

I'm not so keen on catch, I think it will cause confusion, and as you say the type becomes uncertain. But I think it deserves a place in the rejected ideas section.

[1st1]

IMO changing the syntax to catch will make the PEP dead in the water. It instantly raises questions like "why do we have two different keywords" etc. I'm very -1 on catch.

One is what I said above about naming except*- is the unpacking analogy valid? The syntax makes it look like we're unpacking the type or type tuple.

That's fair, I did stretch the analogy there a bit. Still, unpacking-like syntax interpretation resonates with me, personally, better.

[gvanrossum]

Does either file already call a _Py function from the other? If not I would put it here, since this is the more low-level file. And the name needs to be changed to something starting with _Py.

[iritkatriel]

I'll make it _Py_WriteIndent.

Followup question:

Before this PR, pythonrun calls PyTraceBack_Print. After this PR it calls the new function PyTraceBack_PrintIndented. PyTraceBack_Print is declared in traceback.h as PyAPI_FUNC(int), but I added PyTraceBack_PrintIndented without that. I guess this means that PyTraceBack_PrintIndented should be _PyTraceBack_PrintIndented (as in private).

[iritkatriel]

I'm working on raise in except* on another branch because it's another layer of complexity. #8

[graingert]

how does this interact with with virtual subclasses? bpo url: https://bugs.python.org/issue12029

currently except* is described as:

The matching of except* clauses against an ExceptionGroup is performed recursively, using the ExceptionGroup.split() method:

and the split method with a type is defined as:

eg.split(TypeError), is equivalent to eg.split(lambda e: isinstance(e, TypeError)).

so from the spec I'd expect that:

    def test_exception_match_calls_subclasscheck(self):
        class SubclassEvery(type):
            def __subclasscheck__(cls, other):
                return other is not ExceptionGroup

            def __instancecheck__(cls, other):
                return cls.__subclasscheck__(type(other))

        class Special(Exception, metaclass=SubclassEvery):
            pass

        try:
            raise ValueError
        except *Special:  # caught
            pass

        try:
            raise ValueError
        except Special:  # not caught
            pass

but the code uses PyErr_GivenExceptionMatches which has the bpo-12029 behaviour

[iritkatriel]

ExceptionGroup cannot be subclassed, and for other exceptions this is exactly the same issue as bpo-12029. Or do you think it is a different issue here?

[graingert]

In this case eg.project(Special), is different from eg.project(lambda e: isinstance(e, Special))

[graingert]

consider an isinstance_gem function that uses PyErr_GivenExceptionMatches:

>>> def isinstance_gem(instance, types):
...     return bool(ctypes.pythonapi.PyErr_GivenExceptionMatches(ctypes.py_object(instance), ctypes.py_object(types)))
... 
>>> isinstance_gem(ValueError(), Special)
False
>>> isinstance(ValueError(), Special)
True

[iritkatriel]

Due to the same (known) issue of bpo-12029, not a new issue introduced in this PEP. Is that correct?

[graingert]

it's either an error in the PEP or an issue in this PR
the PEP reads:

eg.split(TypeError), is equivalent to eg.split(lambda e: isinstance(e, TypeError)).

if it's an error in the pep then it should instead be:

def given_exception_matches(given, exc):
    try:
        raise given
    except exc:
        return True
    except:
        return False

eg.split(TypeError), is equivalent to eg.split(lambda e: given_exception_matches(e, TypeError)).

otherwise should this code be changed to call PyObject_IsInstance?

[iritkatriel]

Ok, I understand thanks.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.