Skip to content

Test tooling: Report line number when expected_results.txt contains format errors #6002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
vmichal wants to merge 2 commits into
base: main
Choose a base branch
from
+8 −8
Draft

Test tooling: Report line number when expected_results.txt contains format errors #6002

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2110b7d
Add line number information to printed error messages.
vmichal Jan 9, 2026
27c11d5
Remove parse_commented_file from public interface of file_parsing.py
vmichal Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions tests/utils/stl/test/file_parsing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def _do_crosslist(ctx: _ParseCtx):


def _parse_env_lst(env_lst: Path, ctx: _ParseCtx):
for line in parse_commented_file(env_lst):
for line_number, line in _parse_commented_file(env_lst):
Copy link
Member

@StephanTLavavej StephanTLavavej Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pylance reports that line_number is unused. Using _ avoids this.

if (m:=_INCLUDE_REGEX.match(line)) is not None:
p = env_lst.parent / Path(m.group("filename"))
_parse_env_lst(p, ctx)
Expand All @@ -112,19 +112,19 @@ def _parse_env_lst(env_lst: Path, ctx: _ParseCtx):
ctx.current.append(_parse_env_line(line))


def parse_commented_file(filename: Union[str, bytes, os.PathLike]) \
-> List[str]:
def _parse_commented_file(filename: Union[str, bytes, os.PathLike]) \
-> List[Tuple[int, str]]:
if str(filename) in _preprocessed_file_cache:
return _preprocessed_file_cache[str(filename)]

filename_path = Path(filename)
result = list()
with filename_path.open() as f:
for line in f.readlines():
for line_number, line in enumerate(f.readlines(), start=1):
if (line:=_COMMENT_REGEX.sub("", line)):
line = line.strip()
if line:
result.append(line)
result.append((line_number, line))

_preprocessed_file_cache[str(filename)] = result
return result
Expand All @@ -136,17 +136,17 @@ def parse_result_file(filename: Union[str, bytes, os.PathLike]) \
return _expected_result_entry_cache[str(filename)]

res = dict()
for line in parse_commented_file(filename):
for line_number, line in _parse_commented_file(filename):
m = _EXPECTED_RESULT_REGEX.match(line)
if m is None:
raise Exception(f'Incorrectly formatted line "{line}" in {filename}.')
raise Exception(f'Incorrectly formatted line {line_number}: "{line}" in {filename}.')
Copy link
Member

@StephanTLavavej StephanTLavavej Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no objection to these changes, but how are they taking effect without my recently added check_libcxx_paths.py finding problems earlier during configuration? (It implicitly expects properly-formatted lines and is strict about seeing FAIL or SKIPPED so if it sees something unexpected like WOOF or SKIP then it thinks the whole line is a filename, and reports a missing file.)

Copy link
Contributor Author

@vmichal vmichal Jan 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy answer (I think obvious to you) - I did not really pay attention to the broader context of all involved python utilities, so I did not see this interaction.

Now, after review, I can see what you're hinting at. I think we should provide my proposed diagnostics in both cases - when running cmake and when running stl-lit. That suggests some degree of merging (at least partially) check_libcxx_paths and file_parsing.py; or at least one calling functionality from the other.

I will try to implement something, probably on Thursday. I think this has a rather low priority since it only affects us. Until then I converted this PR to draft. Feel free to close, if you prefer.

prefix = m.group("prefix")
result = m.group("result")
result_code = getattr(lit.Test, result, None)
if result_code is None:
result_code = getattr(stl.test.tests, result, None)
if result_code is None:
raise Exception(f'Unknown result code "{result}" in line "{line}" in {filename}.')
raise Exception(f'Unknown result code "{result}" in line {line_number}: "{line}" in {filename}.')
res[prefix] = result_code

_expected_result_entry_cache[str(filename)] = res
Expand Down