Skip to content

Latest commit

 

History

History
109 lines (76 loc) · 4.33 KB

README.md

File metadata and controls

109 lines (76 loc) · 4.33 KB
Raw

Contextual Coverage Example (.NET)

This is the code from the article: https://medium.com/scopedev/context-based-code-coverage-now-with-more-net-d99cb48971bb

The requirements to run this example is to have installed .NET Core 3.1 sdk.

Run Unit Tests with Contextual Coverage

To run the tests using the contextual coverage collector of Scope, execute the following:

dotnet test --collect ScopeAgent

This will run the test suite and create one coverage file for each test in the internal Scope format:

Coverage files
ContextualCoverage.Tests/bin/Debug/netcoreapp3.0/coverage_CachedWeatherTest.json
ContextualCoverage.Tests/bin/Debug/netcoreapp3.0/coverage_FactorialTest.json
ContextualCoverage.Tests/bin/Debug/netcoreapp3.0/coverage_FibonacciTest.json
ContextualCoverage.Tests/bin/Debug/netcoreapp3.0/coverage_WeatherTest.json

Run an ASP.NET Web Api with Request based contextual coverage

In order to run the ASP.NET Web Api example first we must compile the ContextualCoverage.Api project then patch the binaries using the ContextualCoverage.Processor project before running the Api.

dotnet build -c Release
cd ContextualCoverage.Processor
./run.sh

Then we run the ContextualCoverage.Api withoud rebuilding:

cd ../ContextualCoverage.Api
dotnet run -c Release --no-build

This will start the Web Api, you can try by pointing the browser to: https://localhost:5001/weatherforecast. This will give you the current weather forecast for London in Fahrenheit.

Now we can enable a coverage of a request by navigating to: https://localhost:5001/weatherforecast?cover

This will create a coverage file for that request in:

Coverage files
ContextualCoverage.Api/coverage.json

Performance

For performance analysis we ran:

wrk -c 8 -d 10 -t 4 https://localhost:5001/weatherforecast

and

wrk -c 8 -d 10 -t 4 https://localhost:5001/weatherforecast?cover

We did that 10 times with 3 possible scenarios:

  1. No Coverage: This represent the original build without the CoverageMiddleware.
  2. With Coverage Disabled: This scenario contains the patched build with the CoverageMiddleware active but without opening a coverage session (not query=cover)
  3. With Coverage Enabled: This scenario has the patched build with the CoverageMiddlewareactive and an active session for the request with query=cover.

The results in a 13 inch MacBook Pro of late 2014 are:

Avg Req/Sec Std. Dev. % Delta
No Coverage 21665.11 385.31 100% -
With Coverage Disabled 20982.86 680.18 96.85% -3.15%
With Coverage Enabled 16990.16 242.84 78.42% -21.58%

The results in a 15 inch MacBook Pro of 2018 are:

Avg Req/Sec Std. Dev. % Delta
No Coverage 48072.39 472.12 100% -
With Coverage Disabled 47939.73 422.92 99.72% -0.28%
With Coverage Enabled 41821.98 711.15 87.24% -12.76%

Using the same machine but, running:

wrk -c 12 -d 10 -t 6 https://localhost:5001/weatherforecast
Avg Req/Sec Std. Dev. % Delta
No Coverage 48459.94 388.34 100% -
With Coverage Disabled 48456.47 376.71 99.99% -0.01%
With Coverage Enabled 46529.14 447.39 96.02% -3.98%

The results in a Ryzen Threadripper 2950X 16-Core Processor (Windows 10 + WSL), running:

wrk -c 32 -d 10 -t 16 https://localhost:5001/weatherforecast
Avg Req/Sec Std. Dev. % Delta
No Coverage 73888.35 1438.84 100% -
With Coverage Disabled 73236.34 2258.05 99.12% -0.88%
With Coverage Enabled 64465.51 1830.63 88.02% -11.98%