short test summary and -vv

[ewjoachim]

a detailed description of the bug or problem you are having

(I've done my best to try and see if this is a duplicate and I'm sorry if it is, but it seems it's not)

Following #11777 (8.2.0) | changelog:

Text is no longer truncated in the short test summary info section when -vv is given.

The short summary can become completely impractical with -vv because each failure message could be hundreds of lines long.

It seems to be a strange choice given #11777 is about the text being truncated in:

E       assert 50 == 49
E        +  where 50 = len({0: 0, 1: 1, 2: 2, 3: 3, ...})

and I don't see the original poster complaining about the short summary, but both were changed?

Was it the intended change ? Is there something we can do to keep the original -vv behaviour for the test output, and keep the "short test summary" short ? It's actually quite useful in CI that the last thing printed is the list of tests that failed and not why each of them failed: it's a summary and the full output is available above anyway.

I think the decision a lot of versions ago to add an excerpt of the failure reason to the short summary was a good compromise: it used otherwise unused space to add more useful info. But if that excerpt start taking more screen real estate space (potentially 100x more) than the summary itself, I'm not sure I understand why we would want that?

Maybe we could introduce a different "letter" (or a different option or something) to -r to get the list of tests without any kind of explanation of why they failed? Maybe the fix for #11777 should have affected the "failure explanation" and not the short summary?

output of pip list from the virtual environment you are using

$ pip list
pip list
Package   Version
--------- -------
iniconfig 2.0.0
packaging 24.1
pip       24.0
pluggy    1.5.0
pytest    8.3.2

pytest and operating system versions

pytest==8.3.2
Ubuntu 22.04.4 LTS

minimal example if possible

def test_foo():
    assert "a\n" * 50 == ""

$ pytest -vv
============================================================== test session starts ===============================================================
platform linux -- Python 3.12.4, pytest-8.3.2, pluggy-1.5.0 -- /home/joachim/other-src/pytest-verbose/.venv/bin/python3.12
cachedir: .pytest_cache
rootdir: /home/joachim/other-src/pytest-verbose
collected 1 item                                                                                                                                 

test_verbose.py::test_foo FAILED                                                                                                           [100%]

==================================================================== FAILURES ====================================================================
____________________________________________________________________ test_foo ____________________________________________________________________

    def test_foo():
>       assert "a\n" * 50 == ""
E       AssertionError: assert 'a\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\n' == ''
E         
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a
E         + a

test_verbose.py:2: AssertionError
============================================================ short test summary info =============================================================
FAILED test_verbose.py::test_foo - AssertionError: assert 'a\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\na\n' == ''
  
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
  + a
=============================================================== 1 failed in 0.01s ================================================================