How to combine tonic-web and axum with a "mini-proxy"

[nguiard]

I'm experimenting with creating a web server that mixes axum and tonic. Running them both in parallel in the same process is fine (on different ports of course). Clients would use both, depending on what they need. However, I'd love to remove the need for an external proxy like Envoy.

So I'd love to have a single entry point (most probably axum), and when clients request on a /grpc prefix, send the request to tonic-web. I know I can probably just actually proxy an HTTP call to tonic with reqwest or similar, passing the full request, and just rewriting the location to remove the /grpc prefix.

However, adding an actual HTTP request seems inefficient since I have everything in the same process. I'd rather call the service more or less directly through tonic-web if it is possible, just skipping the additionnal TCP connection. How can I approach this problem?

I see tonic::Request has a from_http method which seems interesting, but I'm not sure that's the way to go as I can't find an equivalent to_http for the return path of the response... Just very unsure on the best approach to solve this.

[nguiard]

I'm pretty sure I found a satisfying solution. I'll describe it here as soon as I get it working as wanted.

[nguiard]

After a bit a research involving some convoluted ways, it turns this is actually very easy. As tonic and tonic-web-wrapped services implement Service from tower-service, you can simply create your axum app and wrap your tonic-web service in axum::routing::any_service. Works great, still answers direct non-web gRPC too (you probably need to have the http2 feature of axum enabled). No need for weird url prefix, rewriting or anything:

    let grpc_web_greeter = tonic_web::enable(GreeterServer::new(GreeterStruct));
    let axum_addr = "[::]:50051".parse()?;
    let web_app = axum::Router::new()
        .route(
            "/greeter.Greeter/*rpc", 
            axum::routing::any_service(grpc_web_greeter.clone())
        )
        .route("/", get(web_root));

Or, if you need a full example: given a greeter.proto file like this:

syntax = "proto3";
package greeter;

service Greeter {
    rpc Greet (GreetRequest) returns (GreetReply);
}

message GreetRequest {
    string name = 1;
}

message GreetReply {
    string answer = 1;
}

You can have a combined grpc + grpc-web + "classic" web service, all on the same port, like so:

use std::error::Error;
use axum::routing::get;

use greeter::{
    greeter_server::{Greeter, GreeterServer},
    GreetReply, GreetRequest,
};

mod greeter {
    tonic::include_proto!("greeter");
}

struct GreeterStruct;

#[tonic::async_trait]
impl Greeter for GreeterStruct {
    async fn greet(
        &self,
        req: tonic::Request<GreetRequest>,
    ) -> Result<tonic::Response<GreetReply>, tonic::Status> {
        let name = req.into_inner().name;
        println!("In greet(), name: {}", name);

        let reply = GreetReply {
            answer: format!("Hello {name}!").into(),
        };

        Ok(tonic::Response::new(reply))
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let grpc_web_greeter = tonic_web::enable(GreeterServer::new(GreeterStruct));
    let axum_addr = "[::]:50051".parse()?;
    let web_app = axum::Router::new()
        .route(
            "/greeter.Greeter/*rpc", 
            axum::routing::any_service(grpc_web_greeter.clone())
        )
        .route("/", get(web_root));

    axum::Server::bind(&axum_addr)
        .serve(web_app.into_make_service())
        .await?;

    Ok(())
}

async fn web_root() -> &'static str {
    "Hello World!"
}

[davidpdrsn]

ConnectInfo is also available on a tonic request's extensions.

[HuakunShen]

What version of axum and tonic were you using?

let web_app = axum::Router::new()
    .route(
        "/greeter.Greeter/*rpc", 
        axum::routing::any_service(grpc_web_greeter.clone())
    )
    .route("/", get(web_root));

On the line axum::routing::any_service(grpc_web_greeter.clone()) I am getting error

the trait bound `CorsGrpcWeb<GreeterServer<GreeterStruct>>: Service<axum::http::Request<axum::body::Body>>` is not satisfied
the trait `Service<tonic::codegen::http::request::Request<tonic::transport::Body>>` is implemented for `CorsGrpcWeb<GreeterServer<GreeterStruct>>`
for that trait implementation, expected `tonic::codegen::http::request::Request<tonic::transport::Body>`, found `axum::http::Request<axum::body::Body>`

Do you know how to fix this?

I am new to axum and tonic. Thanks!

[HuakunShen]

Problem solved. Seems like I just need to change the version of axum to the same version used in tonic.

[JamieMair]

For anyone in the future who is using tonic v0.12 and axum v0.7, the above has issues compiling unless you downgrade a lot of packages. I also couldn't find a way to get cors to work with the above method. Instead you can modify the example to convert from a tonic Server into an axum Route (use the implementation in the above but replace the main function):

use axum::routing::get;
use tonic::transport::Server;
use tower_http::cors::{Any, CorsLayer};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let grpc_web_greeter = tonic_web::enable(GreeterServer::new(GreeterStruct));
    let axum_addr = "[::]:50051".parse()?;

    let cors = CorsLayer::new()
        .allow_headers(Any)
        .allow_methods(Any)
        .allow_origin(Any);

    let app = Server::builder()
        .layer(cors)
        .add_service(tonic_web::enable(grpc_web_greeter))
        .into_router() // Can add additional routes here with axum
        .route("/", get(web_root));

    let listener = tokio::net::TcpListener::bind(addr).await?;
    axum::serve(listener, app).await?;

    Ok(())
}

[anotherchudov]

unfortunately, it appears that any layers before the .add_service call are ignored.
i.e.

let app = Server::builder()
        .accept_http1(true)
        .layer(cors) // will never be called
        .layer(GrpcWebLayer::new())
        .add_service(grpc_web_greeter)
        .into_router();