MockWebServer - simulate lower-level DNS/connection/TLS/HTTP Protocol issues

[cloudshiftchris]

There's classes of failures below the "HTTP message" level that are hard to unit-test; this would help improve the robustness of the OkHttp client and software that builds on it (e.g. frameworks, SDKs).

Proposal: model the lower-level components in MockWebServer such that their behaviour can be adjusted.

Key areas:

• DNS lookups (simulate DNS latency, failures)
• Connection establishment (simulate connection latency, establishment failures, closing of connections at inopportune times)
• TLS negotiation (exposed ciphers, handshake latency, protocol violations)
• HTTP (/2) (protocol latency, protocol violations) (NOTE: this refers to the protocol level, not messages sent over HTTP)

A rough sampling of related issues that may be easier to prevent / diagnose with enhanced testability:

Connection issues
* awslabs/aws-sdk-kotlin#1214

HTTP (/2) protocol violations
* #8524
* #8408
* #7913
* #7840
* #3955

DNS failures
* #8481

[yschimke]

I'd suggest something like mockserver for this. OkHttp MockWebServer is not being actively enhanced.

https://www.mock-server.com/

[cloudshiftchris]

Was hoping part of this would be for OkHttp to improve testability/quality, whether that be with MockWebServer or otherwise.

[yschimke]

Do you have a concrete example of one of these tests? It's potentially something we would add as tests, but unlikely to provide new public API in MockWebServer. There are already better more complete mock servers out there.

For DNS, this is already possible by providing a custom Dns to the client.

For http2 frames, this is already possible internally with some examples tests.

[cloudshiftchris]

A few examples:

1. The adventure in IO Exception when listing s3 objects in bucket awslabs/aws-sdk-kotlin#1214 could be better tested to ensure connection pool / stale / closed connection / retry semantics are correct (they are questionable atm):

• Close idle connections / streams after short period of inactivity (< 10 seconds); including testing with multiple connections in the pool;
• Close connection/stream upon receipt of a message, to simulate server/software crashing (or otherwise faulting), closing the connection before sending a response;

2. Each of these look to be HTTP/2 protocol violations that could be tested (and perhaps are) to ensure OkHttp behaves correctly:

• Expected a SETTINGS frame but was WINDOW_UPDATE #8524
• Huffman : Fatal Exception: java.lang.NullPointerException #8408
• HTTP/2 RST_STREAM is sent twice after timeout for the same stream #7913
• HTTP/2 does not handle all possible errors #7840
• okhttp3.internal.http2.StreamResetException: stream was reset: PROTOCOL_ERROR/ CANCEL #3955

[yschimke]

Are you looking to get unblocked to add some tests? Or is this just a request for the maintainers to add more tests for issues you've been negatively affected by?

[cloudshiftchris]

A combination of built-in testing for issues we've been negatively affected by and a general desire to "make OkHttp better" where possible/practical.

[yschimke]

Sounds good. for internal project use, if there are things that you might want to add to MockWebServer, then it sounds reasonable. I think we'd just avoid new public API.