Confluent 7.6.2 Schema Registry enabled with Mutual TLS Authentication fails with PKIX path building failed / unable to find valid certification path to requested target

[vpeddada-tibco]

Hi Team,

We have recently upgraded the kafka-schema-registry-client (Java)from 5.2.1 to 7.6.0 and started to observe issues with Mutual TLS Authentication with confluent 7.6.2 version. The following is the exception stack trace for your reference.

2024-07-30T17:10:29,377 ERROR [pool-35-thread-1] io.confluent.kafka.schemaregistry.client.rest.RestService - Failed to send HTTP request to endpoint: https://.compute.amazonaws.com:8081/subjects/schema2-value/versions?normalize=false
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:376)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:319)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:478)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:456)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:199)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1383)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1296)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:416)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:388)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:576)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:201)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246)
at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:304)
at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:409)
at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:589)
at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:577)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:312)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerWithResponse(CachedSchemaRegistryClient.java:428)
at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerWithResponse(CachedSchemaRegistryClient.java:405)
at io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe.registerWithResponse(AbstractKafkaSchemaSerDe.java:513)
at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:118)
at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:68)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1000)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:947)
at com.tibco.bw.palette.kafka.runtime.SendActivity$SendActivityExecutor.sendMessage(SendActivity.java:1415)
at com.tibco.bw.palette.kafka.runtime.SendActivity$SendActivityExecutor.sendMessages(SendActivity.java:1335)
at com.tibco.bw.palette.kafka.runtime.SendActivity$SendActivityExecutor.run(SendActivity.java:1067)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

FYI, We are using Open SSL Self Signed certificates in our system integration environments which were working perfectly fine with older java client 5.2.1.

We tried various approaches to import the certs into the local system truststore CA certs, Java Home CACerts etc but none of these resolved this issues.
Hence we want to check if some security upgrades which may had happened in 7.6.0 version is blocking the TLS Authentication using Self Signed certificates.

   Here are the details for your reference:

kafka-schema-registry-client Version: 7.6.0 (Java)
Confluent Version: 7.6.2
Java: Java 11
OS: Microsoft Windows 11

Thanks for your support and look forward to hear from you soon.