-
Notifications
You must be signed in to change notification settings - Fork 10
Description
One difficulty with the current design is that the connectconformance
test runner must spawn the implementation under test. Since users don't have control over starting the process, it can make it very difficult to debug the process, in order to understand what's going on when there are hard-to-diagnose errors in their RPC framework.
One easy-to-implement idea would allow users to spawn the server and client themselves. In this scenario, the test runner isn't actually running, so none of the results are being validated. It just allows them to feed the test cases to their client implementation or spin up a client that sends the test RPCs to their server implementation, so they can set breakpoints and do other troubleshooting with a debugger. The facilities that the connectconformance
tool must provide to make this work would be a mode where it just generates the files to send to client and server processes' stdin, but doesn't actually launch any processes. We'd also need a way for users to launch the reference clients and servers as a stand-alone process, but in reference mode (this could initially be as easy as adding a --reference
flag to all four of these programs; but since the release artifacts don't include these other binaries, we likely also need a way to expose this capability via the connectconformance
CLI).
A more sophisticated idea might be an "interactive" mode -- each time the runner wants to spawn a program, instead of actually starting it, it can print to the user that they should start the process. The runner would provide paths of files to use as stdin and stdout for the debugged program. It would then await user input for when that program is done (such as pressing enter, or maybe a simple input like having them type "y" if program completed successfully or "n" if it failed).