Trying to connect two services inside GCP, need help. Setting tls programmatically.

[mpiorowski]

So i am trying to connect two services running as Google Cloud Run.

I have a valid connection NodeJS Server -> Rust Client, but cannot make a connection between Rust Server -> Rust Client.
I am getting error:
Error: status: Unknown, message: "transport error", details: [], metadata: MetadataMap { headers: {} }

So e few thing i think may be related:

1. I need to add authorization header, and right now this is a way i am doing it:

let request = Request::from_parts(
    metadata.clone(),
    Default::default(),
    UserId {
        user_id: note.user_id.to_owned(),
    },
);

Where metadata is:
MetadataMap { headers: {"authorization": "Bearer xxxx" } }

2. On nodejs server i needed to create ssl credentials, and i don't see anything related here in rust. Maybe that's the problem?

[mpiorowski]

I think i am missing tls/ssl setup, but what i did

let channel = channel_users
            .tls_config(tonic::transport::ClientTlsConfig::new())
            .context("Failed to create tls config to users service")?
            .connect()
            .await
            .context("Failed to connect to users service")?;

Throw me an error

error trying to connect: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

[mpiorowski]

So this is how gcp do is using go:

func NewConn(host string, insecure bool) (*grpc.ClientConn, error) {
        var opts []grpc.DialOption
        if host != "" {
                opts = append(opts, grpc.WithAuthority(host))
        }

        if insecure {
                opts = append(opts, grpc.WithInsecure())
        } else {
                // Note: On the Windows platform, use of x509.SystemCertPool() requires
                // go version 1.18 or higher.
                systemRoots, err := x509.SystemCertPool()
                if err != nil {
                        return nil, err
                }
                cred := credentials.NewTLS(&tls.Config{
                        RootCAs: systemRoots,
                })
                opts = append(opts, grpc.WithTransportCredentials(cred))
        }

        return grpc.Dial(host, opts...)
}

And i will try do something similiar using https://github.com/est31/rcgen

[mpiorowski]

Next step using rcgen, still not working :( Now i am getting: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

        let subject_alt_names = vec![
            "xxx".to_string(),
            "localhost".to_string(),
        ];
        let cert = generate_simple_self_signed(subject_alt_names)
            .unwrap()
            .serialize_pem()
            .unwrap();
        let server_cert = cert.as_bytes();
        let tonic_cert = Certificate::from_pem(server_cert);
        let tls = ClientTlsConfig::new()
            .ca_certificate(tonic_cert)
            .domain_name("xxxx");
        let channel = channel_users
            .tls_config(tls)
            .context("Failed to create tls config to users service")?
            .connect()
            .await
            .context("Failed to connect to users service")?;

[mpiorowski]

This is a perfect example of what i am struggling with ;p
https://jessitron.com/2022/11/02/make-https-work-on-grpc-in-rust-load-a-root-certificate-into-the-tls-config/

[mpiorowski]

All i need to do was to add "tls-roots" to features........omg, such a simple solution......
And everything works with one simple line:
let users_conn = UsersServiceClient::connect("xxxx").await?;

[wspeirs]

I have that in my client Docker file as well:

FROM debian:12-slim

RUN apt update
RUN apt install -y ca-certificates

I have enabled my server to listen on the internet, and can hit it from my machine via grpc_health_probe, but still fails during the startup of my client container. I was going over a VPC, but have disabled all of that to just try and get this working. Def going to make a blog post somewhere when I finally crack this :-)

[Pascualex]

Haha it's certainly very weird. I would try to run the client from my PC without Docker (enabling public access for the gRPC server) just to discard any error that can occur on the building and deployment stages.

[wspeirs]

I got it working with the gRPC server on the Internet, and adding tls-webpki-roots on the client side. However, I think my real issue was that gcloud run services replace service.yaml wasn't actually doing anything, just redeploying the same broken image. I notice that println messages I was putting in my client code weren't changing... it tough to get something working when you aren't actually changing the code :-( Thanks for the help!

[austin362667]

I think it works on these features mentioned because they handle the whole SSL handshake, while tls flag doesn't, meaning tls just handle the first certificate on chain of certificates.

[andreban]

I was running into this, but calling connect() on the client doesn't work because I need to add an interceptor that injects the authorization token, which needs its own channel.

What solved for me was adding the tls-roots feature to tonic , which adds with_native_roots() to ClientTlsConfig, and calling that solves the issue.

Here's how my implementation looks like:

    let token = ""; // Load your GCP token here. The `gcp_auth` create is working well for me.
    let bearer_token = format!("Bearer {}", token.as_str());
    let header_value: MetadataValue<_> = bearer_token.parse()?;

    let http_endpoint = format!("https://datastore.googleapis.com");
    let tls_config = ClientTlsConfig::new().with_native_roots();
    let channel = Channel::from_shared(http_endpoint)?
        .tls_config(tls_config)?
        .connect()
        .await?;

    let mut service = DatastoreClient::with_interceptor(channel, |mut req: Request<()>| {
        req.metadata_mut()
            .insert("authorization", header_value.clone());
        Ok(req)
    });