Introduce plist.err and parse --status

[barnabasdomozi]

Currently, it is a problem to determine if the analysis was successful or failed during execution.

With this patch, we will create plist.err files to indicate a failed analysis action.
The parse --status command was also introduced to
get the analysis summary of an existing report directory.

[bruntib]

Aren't .err files generated only when analysis fails?
Why clangsa and clang-tidy are hard-coded when printing stats?

[barnabasdomozi]

@bruntib

Aren't .err files generated only when analysis fails?

If the return_code is 0 we return from error_file.py's create() method and no .plist.err will be generated.

Why clangsa and clang-tidy are hard-coded when printing stats?

Is there a list of supported analyzers defined somewhere else?
I think in the regex pattern we should try to match for existing analyzers and skip the rest of the files.

[barnabasdomozi]

@bruntib

I added error file generation for the other analyzers as well.

[dkrupp]

Nice work. I have some remarks after testing.
1)
Looks like the missing/failed/outdated/up-to-date statistics are ok

CodeChecker parse --status ./reports -e json
json return value however does not include the missing, failed, up-to-date file list.
soe the missing:[], up-to-date:[] etc. file lists are missing.

It is okay to leave them out from the CodeChecker parse --status ./reports textual output.

The normal textual output should also contain these missing/up-todate/failed etc. status count numbers.

CodeChecker parse --status ./reports
[INFO 2025-11-28 18:35] - ----==== Summary ====----
[INFO 2025-11-28 18:35] - Successfully analyzed
[INFO 2025-11-28 18:35] -   clangsa: 2
[INFO 2025-11-28 18:35] -   clang-tidy: 2
[INFO 2025-11-28 18:35] -   cppcheck: 2
[INFO 2025-11-28 18:35] - Failed to analyze
[INFO 2025-11-28 18:35] - Total analyzed compilation commands: 2
[INFO 2025-11-28 18:35] - Total available compilation commands: 2
[INFO 2025-11-28 18:35] - ----=================----

In the normal txt output when calling status with an invalid file, the output should at least warn the user that the source is not in the compilation database.

CodeChecker parse --status ./reports --file ../xmltest.cpp_no_such_file
[INFO 2025-11-28 18:37] - ----==== Summary ====----
[INFO 2025-11-28 18:37] - Successfully analyzed
[INFO 2025-11-28 18:37] - Failed to analyze
[INFO 2025-11-28 18:37] - Total analyzed compilation commands: 0
[INFO 2025-11-28 18:37] - Total available compilation commands: 0
[INFO 2025-11-28 18:37] - ----=================----

from this, the user will not understand that there is no analysis result for the requested file.

I corrupted one source file file xmltest.cpp, but looks like gcc and cppcheck thinks the analysis result is ok.
I think the error of these analyzers are not handled.

(CodeChecker venv-dev) ednikru@seliiuvd02700[18:45][test-projects/tinyxml2/build]$ CodeChecker parse --status ./reports -e json
{
"analyzers": {
"clangsa": {
"summary": {
"up-to-date": 1,
"outdated": 0,
"missing": 0,
"failed": 1,
"successful": 1
}
},
"clang-tidy": {
"summary": {
"up-to-date": 1,
"outdated": 0,
"missing": 0,
"failed": 1,
"successful": 1
}
},
"cppcheck": {
"summary": {
"up-to-date": 2,
"outdated": 0,
"missing": 0,
"failed": 0,
"successful": 2
}
},
"gcc": {
"summary": {
"up-to-date": 2,
"outdated": 0,
"missing": 0,
"failed": 0,
"successful": 2
}

[barnabasdomozi]

CodeChecker parse --status ./reports -e json
json return value however does not include the missing, failed, up-to-date file list.
soe the missing:[], up-to-date:[] etc. file lists are missing.

This was an intentional decision but it can be changed.
Currently if someone wants the file lists as well, they need to use the --detailed flag:

CodeChecker parse --status --detailed ./reports -e json

The normal textual output should also contain these missing/up-todate/failed etc. status count numbers.

Implemented in commit: 568388d

In the normal txt output when calling status with an invalid file, the output should at least warn the user that the source is not in the compilation database.

Fixed in commit: 568388d

I corrupted one source file file xmltest.cpp, but looks like gcc and cppcheck thinks the analysis result is ok.
I think the error of these analyzers are not handled.

I investigated this an it's related how we handle analyze failures.

Example file:

#include <iostream>

int main()
{
  return 0 // compile error, missing ;
}

When we analyze this file with clangsa or clang-tidy, we get an error code of 1 which indicates an error.
In case of cppcheck, it unfortunately returns with error code 0 meaning successful execution.
Cppcheck's output is also not helpful in this case, it has no indication that an error occurred:

cppcheck ./a.cpp
Checking a.cpp ...

[dkrupp]

Thanks.

For me the --detailed switch does not emit the file level status list.

CodeChecker parse ./reports --status --detailed
[INFO 2025-12-01 14:29] - ----==== Summary ====----
[INFO 2025-12-01 14:29] - Successfully analyzed
[INFO 2025-12-01 14:29] - clangsa: 2
[INFO 2025-12-01 14:29] - clang-tidy: 2
[INFO 2025-12-01 14:29] - cppcheck: 2
[INFO 2025-12-01 14:29] - gcc: 2
[INFO 2025-12-01 14:29] - Failed to analyze
[INFO 2025-12-01 14:29] - Up-to-date analysis results
[INFO 2025-12-01 14:29] - clangsa: 2
[INFO 2025-12-01 14:29] - clang-tidy: 2
[INFO 2025-12-01 14:29] - cppcheck: 2
[INFO 2025-12-01 14:29] - gcc: 2
[INFO 2025-12-01 14:29] - Outdated analysis results
[INFO 2025-12-01 14:29] - Missing analysis results
[INFO 2025-12-01 14:29] - infer: 2
[INFO 2025-12-01 14:29] - Total analyzed compilation commands: 2
[INFO 2025-12-01 14:29] - Total available compilation commands: 2
[INFO 2025-12-01 14:29] - ----=================----

[dkrupp]

The --detailed flag does not show the list of files.

Can you also please add some basic test cases?

[dkrupp]

I would also find it useful if you could add the analys summary automatically at the end of the normal

CodeChecker parse ./report_dir command

Can you pls do that?

[barnabasdomozi]

@dkrupp

The --detailed flag does not show the list of files.

It was implemented for the JSON view (-e json), but now I added to the regular summary message as well: a502e03

I would also find it useful if you could add the analys summary automatically at the end of the normal
CodeChecker parse ./report_dir command

Implemented in latest commit: 367e956

[dkrupp]

LGTM

[dkrupp]

Fixes #4100