Once again about untested files

[a1div0]

I have a project with the following structure:

• .rocks/
• app/
• roles/
• slow_log.lua
• tracing.lua
• const.lua
• vhard_bootstrapper.lua
• demo/
• doc/
• tmp/
• test/
• make_version.lua

I want to test everything that is not in test/ and .rocks/. I write in .luacov:

exclude = {
	'.rocks',
	'test',
}

The result is:

==============================================================================
Summary
==============================================================================

File                        Hits Missed Coverage
------------------------------------------------
app/vshard_bootstrapper.lua 16   2      88.89%
app/roles/slow_log.lua      105  2      98.13%
app/const.lua               4    0      100.00%
------------------------------------------------
Total                       125  4      96.90%

Question: where is tracing.lua?

What is code coverage?

Code coverage is an indicator of how much of the source code is covered by tests.

They tried to solve this problem in #70 (#70). For obvious reasons, this is done with a flag.
Although in my opinion it would be worth making a flag for backward compatibility, that is, turning on the flag if you want the same behavior as before.
And by default, so that the calculation is correct. Okay, let's assume it is.

includeuntestedfiles = true

exclude = {
	'.rocks',
	'test',
}

I run the same test. The result is:

==============================================================================
Summary
==============================================================================

File                        Hits Missed Coverage
------------------------------------------------
app/roles/slow_log.lua      0    80     0.00%
app/roles/tracing.lua       0    90     0.00%
make_version.lua            0    47     0.00%
app/const.lua               4    0      100.00%
app/vshard_bootstrapper.lua 18   0      100.00%
------------------------------------------------
Total                       22   217    9.21%

Great, we see tracing.lua. But where did the coverage percentage of slow_log.lua go?

After studying the documentation it becomes clear that all directories to be checked must be specified. Why existing include/exclude filters cannot be taken into account is a profound mystery.

includeuntestedfiles = {
	'app/',
	'app/roles/',
}

exclude = {
	'.rocks',
	'test',
}

The result is the same (strange, it should already be the correct result):

==============================================================================
Summary
==============================================================================

File                        Hits Missed Coverage
------------------------------------------------
app/roles/slow_log.lua      0    80     0.00%
app/roles/tracing.lua       0    90     0.00%
app/vshard_bootstrapper.lua 16   2      88.89%
app/const.lua               4    0      100.00%
------------------------------------------------
Total                       20   172    10.42%

The correct result could not be obtained. Let's assume. What will happen if we remove the exclude filter, since the includeuntestedfiles filter clearly specifies which files to output?

==============================================================================
Summary
==============================================================================

File                               Hits Missed Coverage
-------------------------------------------------------
app/roles/slow_log.lua             0    80     0.00%
app/roles/tracing.lua              0    90     0.00%
app/vshard_bootstrapper.lua        0    16     0.00%
test/helpers/treegen.lua           19   31     38.00%
test/utils.lua                     24   30     44.44%
test/helpers/utils.lua             26   16     61.90%
test/helper.lua                    21   10     67.74%
test/helpers/cluster.lua           124  49     71.68%
test/helpers/server.lua            98   38     72.06%
test/helpers/migrate.lua           21   7      75.00%
test/helpers/integration.lua       62   16     79.49%
test/integration/slow_log_test.lua 74   3      96.10%
app/const.lua                      4    0      100.00%
test/path.lua                      10   0      100.00%
-------------------------------------------------------
Total                              483  386    55.58%

What we see:

• The output includes files that are not in the list.
• Incorrect result for slow_log.lua.

Here is the option:

include = {
	'app%/*',
	'app%/roles%/*',
}

includeuntestedfiles = {
	'app/',
	'app/roles/',
}

It also does not give the correct result:

==============================================================================
Summary
==============================================================================

File                        Hits Missed Coverage
------------------------------------------------
app/roles/slow_log.lua      0    80     0.00%
app/roles/tracing.lua       0    90     0.00%
app/const.lua               4    0      100.00%
app/vshard_bootstrapper.lua 18   0      100.00%
------------------------------------------------
Total                       22   170    11.46%

And once again, to be sure, we repeat the tests with the following settings:

include = {
	'app%/*',
	'app%/roles%/*',
}

The result for slow_log.lua is now correct. But tracing.lua is missing again.

==============================================================================
Summary
==============================================================================

File                        Hits Missed Coverage
------------------------------------------------
app/vshard_bootstrapper.lua 16   2      88.89%
app/roles/slow_log.lua      105  2      98.13%
app/const.lua               4    0      100.00%
------------------------------------------------
Total                       125  4      96.90%

Even if there are some nuances that I did not take into account, in my opinion the tool should not work like this.
Of course, I may be wrong, but in my opinion this tool should work like this:

• by default, consider all *.lua files that pass include/exclude filters
• additional filters are not needed, they only complicate the operation of the tool.