Plain interfaces without connect.Request / connect.Response

[tux21b]

I am currently evaluating the usage of connect-go. I understand the advantages of having generics for request and response objects which allow accessing HTTP headers and I also understand that they are required for streaming.

In my use-case however, about 99% of the calls are non-streaming and require no access to HTTP headers. In fact, since I would like to use connect-go in the application layer and I do not want to have things like HTTP headers (or even connect-go) in the application layer, I would prefer to be just able to implement the raw service interfaces (similar to GRPC). This would it also make it easier to call functions from non-HTTP contexts (e.g. a CLI) or other functions, since someone does not have to create fake connect.Requests to do that.

So, basically I would like something like:

• the creation of a "raw service interface" that maps the GRPC service definition directly (without any connect.Request / Response wrappers)
• this "raw service interface" should provide no access to HTTP headers and should be easy to implement and use (i.e. no need to fake connect.Requests when calling a function)
• this "raw service interface" should be placed directly in the proto package and should not depend on connect-go
• streaming calls should be (if supported at all) also be abstracted with some kind of message sender interface that does not depend on connect-go. It would be great if it the message sender interface is self contained in the generated proto package.
• connect-go should generate separate *connect packages with interfaces that uses connect.Request / Response wrappers as it is doing right now
• connect-go should also generate a function in the *connect package which adapts a "raw service" (without connect.Request / Response wrappers) into a connect service (with connect.Request / Response wrappers).

What do you think? Is there any other way to achieve a similar result?

[akshayjshah]

I can understand why you might want this, especially if you like layered architecture. I'll spend a little time prototyping in protoc-gen-connect-go, exploring some reflection-based alternatives, and generally seeing what feels reasonable.

We had a fairly elaborate version of this earlier in Connect's development: it went even further than you're suggesting and allowed you to choose on a method-by-method basis whether you'd rather implement the raw or generic version of each RPC. We dropped it because it required quite a bit more code generation, and feedback from early users was that the generics weren't a problem.

If you'd like to get started with something today, you can use a generic function to convert any single func from raw to generic:

func generify[Req any, Res any](
	f func(context.Context, *Req) (*Res, error),
) func(context.Context, *connect.Request[Req]) (*connect.Response[Res], error) {
	return func(ctx context.Context, req *connect.Request[Req]) (*connect.Response[Res], error) {
		res, err := f(ctx, req.Msg)
		if err != nil {
			return nil, err
		}
		return connect.NewResponse(res), nil
	}
}

That helps a little, but you'll still end up writing a lot of boilerplate structs. It's possible that some of the boilerplate could be removed with a combination of Go and Protobuf reflection, but I'd need to spend some time figuring out how.

[tux21b]

Thanks for your elaborate reply!

I also thought about a reflection based solution (and maybe there is a reflection based way to convert whole services at once) but I have dismissed the idea quickly since it will mean that developers had to guess the interface and deal with obscure runtime based reflection errors instead of using the "implement this interface" feature right from their IDE. So I think there is value in having the plain interface. The conversion could then be done with reflection based code of course.

But to be honest, I do not really have an elegant solution to this problem either. Currently the extremely buf dependent interfaces are a deal breaker for us. GRPC doesn't have this problem, but has several other quirks when used in a browser environment. Hopefully we can figure out a solution together.

Have a nice holiday!

[yurifedoseev]

Hi, I'd like to add that this generics is a stopper for the easy switch to connect-go. I have a project which used 3 transport implementation for grpc: pure grpc, improbable-eng/grpc-web and gprc over web-rtc data channel. All transports supports the classic service interfaces in backend. It allows to switch transport or support multiple transports per one service.

[jhump]

@yurifedoseev, if you are looking to use Connect but using the interfaces generated by protoc-gen-grpc-go, take a look at https://github.com/connectrpc/vanguard-go. It can act as middleware that translates protocols. In particular, if you use the vanguardgrpc package therein, it lets you use the existing gRPC-generated interfaces, registered with a *grpc.Server, and then wrap that in a *vanguard.Handler that can expose the service via Connect, gRPC-Web, and gRPC protocols. In fact, if you use HTTP transcoding annotations in your services (like for use with grpc-gateway), it can even handle REST+JSON requests.

[singh-hovr]

What you think about creating the wrapper methods with option provided? This is a deal breaker honestly. Any updates on this? is vanguard a good solution for this problem? is there any tradoffs for using vanguard? like perf? etc

[mattrobenolt]

I don't have a ton of skin in the game here, but I'm generally curious why the current API is a deal breaker. The current API being explicitly accepting a wrapped Request rather than just the raw struct is a valuable thing to me personally. Without that, it hinders a lot of the functionality that I value in Connect over gRPC. It's fundamentally the only way to attach additional headers without the weird APIs that gRPC exposes. This ends up being a lot more natural compared to net/http.

So I'm just curious why this is a deal breaker for you considering you could just write your own shim.

[yurifedoseev]

@mattrobenolt I can add my view.
The main desire is to separate the code from specific framework details and match the standard.
GRPC has a standard representation in golang with the semantic ‘method(ctx, req) (res, error)’
My current project (gaming with webrtc) is using grpc heavily with different transports:

• a standard grpc via http/2 between backends
• a web grpc via http1.1/2 and web sockets
• grpc over unix socket for daemon’s communication
• grpc over web-rtc data channels for low-latency control

Cool part that all these grpc services implement the same grpc service interface. All frameworks details are used only during the setup of the application. It allows to make the code more ubiquitous and reuse methods via different transport protocols.
It’d be nice to continue so support this interfaces with just replacing the transport/config