Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spring Security IPv6 issue - is there a global config setting? #16337

Open
dreamstar-enterprises opened this issue Dec 23, 2024 · 5 comments
Open
Labels
status: waiting-for-triage An issue we've not yet triaged type: bug A general bug

Comments

@dreamstar-enterprises
Copy link

dreamstar-enterprises commented Dec 23, 2024

Ok, after 2 days of trouble shooting, I've narrowed this down to being a Spring Security issue.

Goal:
To run secure login page, via a Spring BFF and Spring Rest API, on AWS (IPV6 stack)
Why IPV6 and not IPV4 - (i) it's a much more modern protocol, and (ii) avoids the $600+ annual cost of a NAT Gateway (for Ipv4)
Started in: July 2024 (I know it's been 6 months)

Environment
AWS
ECS - Fargate
Essentially running Docker Images - 2 of them, one for each App
The containers run in an AWS Private Subnet, wrapped around an AWS Security Groups (for better security)
I have an Ipv6 egress only gateway on the route tables in the private subnet (so Ipv6 calls can get out into the internet)

Observations:
Initially my Rest API was unable get an Auth0 public key, as it was timing out.
I found out that this was because my code was resolving to Ipv4, despite me configuring some settings in Java / Spring to prefer Ipv6.
In the end I had to write this code, to get it to work
Note that when calling the webclient I do use a blocking call (not recommended for Reactive non-blocking Spring Webflux), but this seems to work for me, and it seemed ok to me since I'm getting the key from Auth0 on an infrequent refresh cycle.

JwtConfig class - in Spring Rest API

/**
 * Configuration class for JWT handling with IPv6 support.
 * Provides JWT decoder configuration with custom IPv6 resolver and caching mechanisms.
 *
 * @property serverProperties Server properties containing Auth0 JWK set URI configuration
 */
@Configuration
internal class JwtConfig(
    private val serverProperties: ServerProperties,
) {

    companion object {
        private val logger = LoggerFactory.getLogger(JwtConfig::class.java)
    }

    /**
     * Custom implementation of JWKSetSource for retrieving and caching JSON Web Key Sets.
     *
     * @property webClient WebClient configured for IPv6 connections
     * @property jwkSetUri URI endpoint for retrieving the JWK set
     */
    private inner class CustomJwkSetSource(
        private val webClient: WebClient,
        private val jwkSetUri: URI
    ) : JWKSetSource<SecurityContext> {
        private var cachedJwkSet: JWKSet? = null

        /**
         * Retrieves the JWK set from the configured endpoint with caching support.
         * Attempts to fetch a fresh JWK set and falls back to cached version if available.
         *
         * @param refreshEvaluator Optional evaluator for cache refresh decisions
         * @param currentTime Current timestamp for cache evaluation
         * @param context Optional security context for the retrieval operation
         * @return JWKSet containing the retrieved or cached key set
         * @throws JwkSetRetrievalException if retrieval fails and no cached version is available
         */
        override fun getJWKSet(
            refreshEvaluator: JWKSetCacheRefreshEvaluator?,
            currentTime: Long,
            context: SecurityContext?
        ): JWKSet {
            try {
                val response = webClient.get()
                    .uri(jwkSetUri)
                    .retrieve()
                    .toEntity(String::class.java)
                    .block(Duration.ofSeconds(30))
                    ?: return handleJwkSetError("Failed to retrieve JWK set - null response")

                val body = response.body
                    ?: return handleJwkSetError("Empty JWK set response")

                return try {
                    JWKSet.parse(body).also {
                        cachedJwkSet = it
                        logger.info("Successfully retrieved and cached new JWK set")
                    }
                } catch (e: Exception) {
                    logger.error("Failed to parse JWK set response", e)
                    handleJwkSetError("Invalid JWK set format: ${e.message}")
                }
            } catch (ex: Exception) {
                logger.error("Error retrieving JWK set", ex)
                return handleJwkSetError("Failed to retrieve JWK set: ${ex.message}")
            }
        }

        /**
         * Handles JWK set retrieval errors by attempting to use cached data.
         * Logs error details and attempts to provide a cached version if available.
         *
         * @param errorMessage Detailed description of the error that occurred
         * @return Cached JWKSet if available
         * @throws JwkSetRetrievalException if no cached data is available
         */
        private fun handleJwkSetError(errorMessage: String): JWKSet {
            cachedJwkSet?.let {
                logger.error("$errorMessage. Falling back to cached version")
                return it
            }

            logger.error("$errorMessage. No cached version available")
            throw JwkSetRetrievalException(errorMessage)
        }

        override fun close() {
            // No resources to clean up
        }
    }

    /**
     * Exception thrown when JWK set retrieval fails and no cached version is available.
     *
     * @param message Detailed error message describing the failure
     */
    private class JwkSetRetrievalException(message: String) : RuntimeException(message)

    /**
     * Creates and configures a ReactiveJwtDecoder bean with IPv6 support.
     * Configures the decoder with:
     * - Custom IPv6 resolver
     * - JWK set caching
     * - Rate limiting
     * - Error handling
     *
     * @return Configured ReactiveJwtDecoder instance
     */
    @Bean
    fun jwtDecoder(): ReactiveJwtDecoder {
        val jwkSetUri = URI(serverProperties.auth0JwKeySetUri)

        logger.info("Initializing JWT decoder for jwkSetUri: $jwkSetUri")

        val jwkSource = JWKSourceBuilder.create(
                CustomJwkSetSource(createIpv6WebClient(jwkSetUri), jwkSetUri)
            )
            .cache(
                TimeUnit.HOURS.toMillis(12),
                TimeUnit.MINUTES.toMillis(60)
            )
            .rateLimited(TimeUnit.MINUTES.toMillis(10)) { event ->
                logger.warn("Rate limit reached for JWK source")
            }
            .refreshAheadCache(true)
            .build()

        val jwtProcessor = DefaultJWTProcessor<SecurityContext>().apply {
            setJWSKeySelector(JWSAlgorithmFamilyJWSKeySelector.fromJWKSource(jwkSource))
        }

        return NimbusReactiveJwtDecoder { jwt ->
            Mono.fromCallable {
                try {
                    jwtProcessor.process(jwt, null)
                } catch (ex: Exception) {
                    logger.error("JWT processing error", ex)
                    throw ex
                }
            }
        }
    }
}

IPV6 Utility class -

For creating the IPV6 web-builder
The most important thing for me to see here is the Resource address and whether that is an IPV6 address:
And in my logs I see this, and when I run this on AWS, all works as expected the Auth0 JwT key is retrieved

2024-12-23T09:00:35.748Z DEBUG 1 --- [Timesheets-RESTApiApplication] [tor-tcp-epoll-2] c.v.t.auth.ipv6.Ipv6WebClientUtil        : Remote Address: dev-ld4xuyx1eigiqoge.uk.auth0.com/[2606:4700:4400:0:0:0:6812:2346]:443
2024-12-23T09:00:36.202860361Z 2024-12-23T09:00:36.201Z  INFO 1 --- [Timesheets-RESTApiApplication] [           main] c.v.t.auth.tokens.jwt.JwtConfig          : Successfully retrieved and cached new JWK set
/**
 * Utility object for IPv6-enabled WebClient creation and DNS resolution.
 * Provides reusable components for IPv6 network connectivity with IPv4 fallback.
 */
internal object Ipv6WebClientUtil {

    private val logger = LoggerFactory.getLogger(Ipv6WebClientUtil::class.java)

    /**
     * Initialize IPv6 preferences when the object is first accessed.
     * These are JVM-wide settings that should be set once at startup.
     */
    init {
        initializeIpv6Preferences()
    }

    /**
     * Initializes IPv6 preferences for the JVM and logs current settings.
     * This should only be called once during application startup.
     */
    private fun initializeIpv6Preferences() {
        // Set IPv6 preferences
        System.setProperty("java.net.preferIPv6Addresses", "true")
        System.setProperty("java.net.preferIPv6Stack", "true")
        java.security.Security.setProperty("networkaddress.preferIPv6Addresses", "true")

        // Log the current settings
        logger.debug("IPv6 Preferences:")
        logger.debug("preferIPv6Addresses: ${System.getProperty("java.net.preferIPv6Addresses")}")
        logger.debug("preferIPv6Stack: ${System.getProperty("java.net.preferIPv6Stack")}")
        logger.debug("Security preferIPv6Addresses: ${java.security.Security.getProperty("networkaddress.preferIPv6Addresses")}")
    }

    /**
     * Creates a custom IPv6 address resolver for Netty.
     * Handles both single and multiple address resolution with IPv6 preference.
     * Falls back to IPv4 if IPv6 is not available
     *
     * @param host The hostname to resolve
     * @return AddressResolverGroup configured for IPv6 resolution
     */
    fun createIpv6Resolver(host: String): AddressResolverGroup<InetSocketAddress> {
        return object : AddressResolverGroup<InetSocketAddress>() {
            override fun newResolver(executor: EventExecutor) =
                object : AbstractAddressResolver<InetSocketAddress>(executor) {
                    override fun doResolve(address: InetSocketAddress?, promise: Promise<InetSocketAddress>?) {
                        try {
                            val hostname = address?.hostName ?: host
                            val port = address?.port ?: 443

                            logger.debug("Attempting to resolve IPv6 address for: $hostname:$port")

                            // First attempt to resolve IPv6 address
                            val resolvedAddress = InetAddress.getAllByName(hostname)
                                .firstOrNull { it is Inet6Address }
                                ?.let {
                                    logger.info("Resolved IPv6: ${it.hostAddress}")
                                    InetSocketAddress(it, port)
                                }
                                ?: run {
                                    // Fallback to IPv4 if no IPv6 address is available
                                    InetAddress.getAllByName(hostname)
                                        .firstOrNull()
                                        ?.let {
                                            logger.info("Resolved IPv4: ${it.hostAddress}")
                                            InetSocketAddress(it, port)
                                        }
                                }
                                ?: throw IllegalStateException("No address available for $hostname")

                            logger.debug("Resolved address: ${resolvedAddress.address.hostAddress}:$port")
                            promise?.setSuccess(resolvedAddress)
                        } catch (ex: Exception) {
                            logger.error("Failed to resolve IPv6 address", ex)
                            promise?.setFailure(ex)
                        }
                    }

                    override fun doResolveAll(address: InetSocketAddress?, promise: Promise<MutableList<InetSocketAddress>>?) {
                        try {
                            val hostname = address?.hostName ?: host
                            val port = address?.port ?: 443

                            // First resolve IPv6 addresses, then fallback to IPv4
                            val addresses = InetAddress.getAllByName(hostname)
                                .filterIsInstance<Inet6Address>()
                                .map { InetSocketAddress(it, port) }
                                .ifEmpty {
                                    InetAddress.getAllByName(hostname)
                                        .map { InetSocketAddress(it, port) }
                                }

                            if (addresses.isEmpty()) {
                                promise?.setFailure(IllegalStateException("No addresses available for $hostname"))
                            } else {
                                promise?.setSuccess(addresses.toMutableList())
                            }
                        } catch (e: Exception) {
                            logger.error("Failed to resolve addresses", e)
                            promise?.setFailure(e)
                        }
                    }

                    override fun doIsResolved(address: InetSocketAddress?) =
                        address?.address != null && !address.isUnresolved
                }
        }
    }

    /**
     * Creates an IPv6-enabled WebClient for token endpoint communications.
     *
     * Features:
     * - IPv6 address resolution with IPv4 fallback
     * - Custom timeout configuration
     * - Secure HTTPS support
     * - Connection logging
     *
     * @param host The token endpoint hostname to resolve
     * @return WebClient configured with IPv6 support
     */
    fun createIpv6WebClient(uri: URI): WebClient {
        return WebClient.builder()
            .baseUrl(uri.toString()) // Use the full URI as base URL
            .clientConnector(
                ReactorClientHttpConnector(
                    HttpClient.from(
                        TcpClient.create()
                            .resolver(createIpv6Resolver(uri.host)) // Resolver uses only the host
                            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
                            .doOnConnected { connection ->
                                val remoteAddress = connection.channel().remoteAddress()
                                logger.debug("Remote Address: {}", remoteAddress)
                            }
                    ).secure()
                )
            )
            .build()
    }

}

THE PROBLEM

Now the problem is with my Spring BFF

In my OauthAuthorizedManagerConfig class, I have the following code. I included the extra bean tokenResponseClient, to see if I could see and also set the webclient when the Spring BFF - Auth0 token exchange is done (to force preference to IPV6)

Here, I use the same Ipv6 Util class as I did in my Spring Rest API, with all the Spring / Java settings to prefer Ipv6.
The specific call to the ipv6 util webclient is here:

   return WebClientReactiveAuthorizationCodeTokenResponseClient().apply {
           setWebClient(createIpv6WebClient(jwkSetUri)

And the Spring BFF class itself...

OauthAuthorizedManagerConfig class

@Configuration
internal class OAuth2AuthorizedManagerConfig(
    private val serverProperties: ServerProperties
) {

    companion object {
        private val logger = LoggerFactory.getLogger(OAuth2AuthorizedManagerConfig::class.java)
    }

    /**
     * Creates and configures the ReactiveOAuth2AuthorizedClientManager.
     *
     * Manager responsibilities:
     * - Coordinates client registration access
     * - Manages token lifecycle
     * - Handles authorization persistence
     * - Provides reactive processing
     *
     * Security measures:
     * - Secure component integration
     * - Token handling procedures
     * - State validation
     * - Operation logging
     *
     * @param reactiveClientRegistrationRepository Source of OAuth2 client registrations
     * @param redisServerOAuth2AuthorizedClientRepository Persistence for authorized clients
     * @param reactiveAuthorizedClientProvider Token lifecycle manager
     * @return Configured [ReactiveOAuth2AuthorizedClientManager]
     */
    @Bean
    fun reactiveAuthorizedClientManager(
        reactiveClientRegistrationRepository: ReactiveClientRegistrationRepository,
        redisServerOAuth2AuthorizedClientRepository: RedisServerOAuth2AuthorizedClientRepository,
        reactiveAuthorizedClientProvider: ReactiveOAuth2AuthorizedClientProvider,
        authoritiesConverter: AuthoritiesConverterClaimSet,
    ): ReactiveOAuth2AuthorizedClientManager = DefaultReactiveOAuth2AuthorizedClientManager(
        reactiveClientRegistrationRepository,
        redisServerOAuth2AuthorizedClientRepository
    ).apply {
        logger.debug("Configuring OAuth2 authorized client manager")

        setAuthorizedClientProvider(reactiveAuthorizedClientProvider)
    }

    @Bean
    fun tokenResponseClient(): ReactiveOAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest> {
        val auth0TokenUri = URI(serverProperties.auth0TokenUri)
        val jwkSetUri = URI(serverProperties.auth0JwKeySetUri)

        logger.info("Initializing Auth0 token for auth0TokenUri: $auth0TokenUri")

        return WebClientReactiveAuthorizationCodeTokenResponseClient().apply {
            setWebClient(createIpv6WebClient(jwkSetUri)
                .mutate()
                .filter { request, next ->
                    logger.debug("""
                    Token Exchange Request:
                    ----------------------------------------
                    URI: ${request.url()}
                    Method: ${request.method()}
                    Headers: ${
                        request.headers().map { (key, value) ->
                            "$key: ${value.joinToString(", ")}"
                        }.joinToString("\n")
                    }
                    Request Time: ${LocalDateTime.now()}
                    Grant Type: ${request.headers().contentType}
                    Request URI: ${request.url()}
                """.trimIndent())

                    next.exchange(request)
                        .doOnNext { response ->
                            logger.debug("""
                            Token Exchange Response:
                            ----------------------------------------
                            Status: ${response.statusCode()}
                            Response Time: ${LocalDateTime.now()}
                            Headers (Full):
                            ${
                                response.headers().asHttpHeaders()
                                    .entries
                                    .joinToString("\n") { (key, value) ->
                                        "  $key: ${value.joinToString(", ")}"
                                    }
                            }
                        """.trimIndent())
                        }
                        .doOnError { error ->
                            logger.error("""
                            Token Exchange Error:
                            ----------------------------------------
                            Error Type: ${error.javaClass.simpleName}
                            Message: ${error.message}
                            Stack Trace: 
                            ${error.stackTraceToString()}
                        """.trimIndent())
                        }
                }
                .clientConnector(ReactorClientHttpConnector(
                    HttpClient.create().resolver(DefaultAddressResolverGroup.INSTANCE)
                    .doOnConnected { connection ->
                        val remoteAddress = connection.channel().remoteAddress()
                        logger.debug("Remote Address: {}", remoteAddress)
                    }
                ))
                .build()
            )
        }
    }
}

Internally when the webclient is set and called by WebClientReactiveAuthorizationCodeTokenResponseClient,

return WebClientReactiveAuthorizationCodeTokenResponseClient().apply {
            setWebClient(createIpv6WebClient(jwkSetUri)

It does something like this (it doesn't use the block command, probably a red herring, but the only difference I noted to my Spring Rest API code)

	private RequestHeadersSpec<?> populateRequest(T grantRequest) {
		return this.webClient.post()
			.uri(clientRegistration(grantRequest).getProviderDetails().getTokenUri())
			.headers((headers) -> {
				HttpHeaders headersToAdd = getHeadersConverter().convert(grantRequest);
				if (headersToAdd != null) {
					headers.addAll(headersToAdd);
				}
			})
			.body(createTokenRequestBody(grantRequest));
	}

But importantly, for some reason, the resolved address always ends up being an Ipv4 one (when using val jwkSetUri = URI(serverProperties.auth0JwKeySetUri) or val auth0TokenUri = URI(serverProperties.auth0TokenUri) as the input URL)

2024-12-22T20:23:22.774Z DEBUG 1 --- [BFFApplication] [or-http-epoll-2] c.v.b.a.m.OAuth2AuthorizedManagerConfig : Remote Address: dev-ld4xuyx1eigiqoge.uk.auth0.com/172.64.152.186:443

And this is the root cause of the issue...

The request won't pass out of the AWS Environment (unless I add a NAT Gateway, to allow outbound Ipv4 requests from a Private Subnet)

Not an Auth0 issue

If I take the JwtConfig class, from my Spring Rest API and place a copy of it in my Spring BFF container, that still resolves to IPV6. (when using val jwkSetUri = URI(serverProperties.auth0JwKeySetUri) as the input URI to createIpv6WebClient() ) - so that rules this out as being an Auth0 issue.

It is more an issue with the tokenResponseClient class, and the way the webClient is being called, which I cannot pin down.

I imagine there might be a few webclients all over the Spring BFF, which call auth0 endpoints here and there (see below), apart from the tokenResponseClient bean. But for some reason, at least the tokenResponseClient bean above it is not resolving to IPV6 - which is causing problems.

Is anyone from the Spring Security team able to provide guidance or help? Maybe they can see an issue with my configuration or code?

Many thanks in advance

Spring BFF Auth0 endpoints

Endpoints being called that may potentially be calling webclients supporting / not preferring IPV6, unless here is a global configuration for this, that needs to be put into a particular class.

private fun auth0Registration(): ClientRegistration {
        return ClientRegistration
            .withRegistrationId(serverProperties.auth0AuthRegistrationId)
            .clientId(clientSecurityProperties.auth0ClientId)
            .clientSecret(clientSecurityProperties.auth0ClientSecret)
            .authorizationGrantType(AuthorizationGrantType.AUTHORIZATION_CODE)
            .redirectUri("${serverProperties.clientUri}/login/oauth2/code/${serverProperties.auth0AuthRegistrationId}")
            .authorizationUri("${serverProperties.auth0IssuerUri}/authorize")
            .tokenUri("${serverProperties.auth0IssuerUri}/oauth/token")
            .jwkSetUri("${serverProperties.auth0IssuerUri}/.well-known/jwks.json")
            .userInfoUri("${serverProperties.auth0IssuerUri}/userinfo")
            .providerConfigurationMetadata(mapOf(
                "issuer" to "${serverProperties.auth0IssuerUri}/",
                "authorization_endpoint" to "${serverProperties.auth0IssuerUri}/authorize",
                "token_endpoint" to "${serverProperties.auth0IssuerUri}/oauth/token",
                "userinfo_endpoint" to "${serverProperties.auth0IssuerUri}/userinfo",
                "end_session_endpoint" to "${serverProperties.auth0IssuerUri}/v2/logout",
                "jwks_uri" to "${serverProperties.auth0IssuerUri}/.well-known/jwks.json",
                "revocation_endpoint" to "${serverProperties.auth0IssuerUri}/oauth/revoke"
            ))
            .userNameAttributeName(IdTokenClaimNames.SUB)
            .scope("openid", "offline_access")
            .clientName("BFF-Server-Auth-0")
            .issuerUri("${serverProperties.auth0IssuerUri}/")
            .build()
    }
@dreamstar-enterprises dreamstar-enterprises added status: waiting-for-triage An issue we've not yet triaged type: bug A general bug labels Dec 23, 2024
@dreamstar-enterprises dreamstar-enterprises changed the title Spring Security IPv6 issue confirmed! Spring Security IPv6 issue - is there a global config setting Dec 23, 2024
@dreamstar-enterprises dreamstar-enterprises changed the title Spring Security IPv6 issue - is there a global config setting Spring Security IPv6 issue - is there a global config setting? Dec 23, 2024
@dreamstar-enterprises
Copy link
Author

dreamstar-enterprises commented Dec 23, 2024

Ok, I know where I may have gone wrong!

This bit here at the end of tokenResponseClient bean was overriding the default resolver, so I removed it from the bottom of the Bean.

   .clientConnector(ReactorClientHttpConnector(
                    HttpClient.create().resolver(DefaultAddressResolverGroup.INSTANCE)
                    .doOnConnected { connection ->
                        val remoteAddress = connection.channel().remoteAddress()
                        logger.debug("Remote Address: {}", remoteAddress)
                    }
                ))

Now I see Ipv6 in the Spring BFF too!

2024-12-23T09:46:28.438Z DEBUG 1 --- [BFFApplication] [tor-tcp-epoll-2] c.v.bff.auth.ipv6.Ipv6WebClientUtil : Remote Address: dev-ld4xuyx1eigiqoge.uk.auth0.com/[2606:4700:4400:0:0:0:6812:2346]:443

Is there a more global setting to ensure preference to Ipv6, as I know that if I don't use my custom resolver it will resolve to ipv4

@dreamstar-enterprises
Copy link
Author

The main issue is with Spring WebClient not defaulting / preferring the use of IPV6 because if I now do this I get an IPV4 address back:

  return WebClientReactiveAuthorizationCodeTokenResponseClient().apply {
            setWebClient(WebClient.builder()
                .baseUrl(jwkSetUri.toString())
                .clientConnector(
                    ReactorClientHttpConnector(
                        HttpClient.create()
                            .doOnConnected { connection ->
                                val remoteAddress = connection.channel().remoteAddress()
                                logger.info("Actually connected to: $remoteAddress")
                            }
                    )
                )

@dreamstar-enterprises
Copy link
Author

dreamstar-enterprises commented Dec 23, 2024

Maybe premature, but this seemed to have solved it for me. I removed my old customipvconfig, and applied it globally, as I think I found the right bean, webClientCustomizer at the very bottom:

/**
 * Global configuration for WebClient instances ensuring IPv6 connectivity preference.
 *
 * This configuration class provides:
 * - JVM-wide IPv6 preference settings
 * - Custom IPv6-preferring DNS resolver
 * - Global WebClient customization
 *
 * The configuration applies to all WebClient instances created via WebClient.builder(),
 * ensuring consistent IPv6 handling across the application with graceful IPv4 fallback.
 */
@Configuration
internal class WebClientGlobalConfig {

    private val logger = LoggerFactory.getLogger(WebClientGlobalConfig::class.java)

    /**
     * Initialize IPv6 preferences when the configuration is created.
     * These are JVM-wide settings that should be set once at startup.
     */
    init {
        initializeIpv6Preferences()
    }

    /**
     * Initializes IPv6 preferences for the JVM and logs current settings.
     *
     * Sets system properties to prefer IPv6 connections and logs the configuration
     * to verify proper initialization. This should only be called once during
     * application startup.
     */
    private fun initializeIpv6Preferences() {
        // Set IPv6 preferences
        System.setProperty("java.net.preferIPv6Addresses", "true")
        System.setProperty("java.net.preferIPv6Stack", "true")
        java.security.Security.setProperty("networkaddress.preferIPv6Addresses", "true")

        // Log the current settings
        logger.debug("IPv6 Preferences:")
        logger.debug("preferIPv6Addresses: ${System.getProperty("java.net.preferIPv6Addresses")}")
        logger.debug("preferIPv6Stack: ${System.getProperty("java.net.preferIPv6Stack")}")
        logger.debug("Security preferIPv6Addresses: ${java.security.Security.getProperty("networkaddress.preferIPv6Addresses")}")
    }

    /**
     * Creates a custom IPv6-preferring address resolver for Netty.
     *
     * This resolver implements the following behavior:
     * 1. Attempts to resolve IPv6 addresses first
     * 2. Falls back to IPv4 if no IPv6 addresses are available
     * 3. Provides detailed logging of address resolution
     * 4. Handles both single and multiple address resolution
     *
     * The resolver maintains non-blocking behavior while performing DNS resolution
     * operations.
     *
     * @return AddressResolverGroup configured for IPv6-first resolution
     */
    fun createIpv6Resolver(): AddressResolverGroup<InetSocketAddress> {
        return object : AddressResolverGroup<InetSocketAddress>() {
            override fun newResolver(executor: EventExecutor) =
                object : AbstractAddressResolver<InetSocketAddress>(executor) {
                    override fun doResolve(inetSocketAddress: InetSocketAddress?, promise: Promise<InetSocketAddress>?) {
                        try {

                            val hostname = inetSocketAddress?.hostName ?: return
                            val addresses = InetAddress.getAllByName(hostname)

                            // First attempt to resolve IPv6 address
                            val ipv6Address = addresses.find { it is Inet6Address }
                            if (ipv6Address != null) {
                                promise?.setSuccess(InetSocketAddress(ipv6Address, inetSocketAddress.port))
                                return
                            }

                            // Fallback to any address (IPv4)
                            val ipv4Address = addresses.firstOrNull()
                            if (ipv4Address != null) {
                                logger.warn("No IPv6 address found for $hostname, falling back to IPv4")
                                promise?.setSuccess(InetSocketAddress(ipv4Address, inetSocketAddress.port))
                            } else {
                                promise?.setFailure(UnknownHostException("No addresses available for $hostname"))
                            }
                        } catch (ex: Exception) {
                            logger.error("Failed to resolve IPv6 address", ex)
                            promise?.setFailure(ex)
                        }
                    }

                    override fun doResolveAll(inetSocketAddress: InetSocketAddress?, promise: Promise<MutableList<InetSocketAddress>>?) {
                        try {
                            val hostname = inetSocketAddress?.hostName ?: return
                            val addresses = InetAddress.getAllByName(hostname)

                            // First try to get all IPv6 addresses
                            val ipv6Addresses = addresses
                                .filter { it is Inet6Address }
                                .map { InetSocketAddress(it, inetSocketAddress.port) }
                                .toMutableList()

                            if (ipv6Addresses.isNotEmpty()) {
                                promise?.setSuccess(ipv6Addresses)
                                return
                            }

                            // Fallback to IPv4 addresses if no IPv6 addresses found
                            logger.warn("No IPv6 addresses found for $hostname, falling back to IPv4")
                            val ipv4Addresses = addresses
                                .map { InetSocketAddress(it, inetSocketAddress.port) }
                                .toMutableList()

                            if (ipv4Addresses.isEmpty()) {
                                promise?.setFailure(UnknownHostException("No addresses available for $hostname"))
                            } else {
                                promise?.setSuccess(ipv4Addresses)
                            }
                        } catch (ex: Exception) {
                            logger.error("Failed to resolve addresses", ex)
                            promise?.setFailure(ex)
                        }
                    }

                    override fun doIsResolved(address: InetSocketAddress?) =
                        address?.address != null && !address.isUnresolved
                }
        }
    }

    /**
     * Provides a global WebClient customizer that enforces IPv6 connectivity with IPv4 fallback.
     *
     * This customizer is automatically applied to all WebClient instances created through
     * WebClient.builder(). It configures:
     * - Custom IPv6-preferring DNS resolver
     * - Connection logging
     * - Proper error handling and fallback behavior
     *
     * Example usage:
     * ```
     * WebClient.builder()
     *     .baseUrl("https://example.com")
     *     .build()
     * ```
     * The resulting WebClient will automatically prefer IPv6 connections.
     *
     * @return WebClientCustomizer that enforces IPv6 preference globally
     */
    @Bean
    fun webClientCustomizer(): WebClientCustomizer {
        return WebClientCustomizer { builder ->
            builder.clientConnector(
                ReactorClientHttpConnector(
                    HttpClient.create()
                        .resolver(createIpv6Resolver())
                        .doOnConnected { connection ->
                            val remoteAddress = connection.channel().remoteAddress()
                            logger.debug("Web Client Connected to: {}", remoteAddress)
                        }
                )
            )
        }
    }
}

@dreamstar-enterprises
Copy link
Author

dreamstar-enterprises commented Dec 23, 2024

This did not work:

Spring Security seems to be bypassing this global WebBuilder customizer, as I cannot see any logs from the implementation below:

@Bean
  fun webClientCustomizer(): WebClientCustomizer {
      logger.debug("Initializing global WebClient customizer with IPv6 resolver")
      return WebClientCustomizer { builder ->
          builder.clientConnector(
              ReactorClientHttpConnector(
                  HttpClient.create()
                      .resolver(createIpv6Resolver())
                      .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
                      .doOnConnected { connection ->
                          val remoteAddress = connection.channel().remoteAddress()
                          logger.debug("Web Client Connected to: {}", remoteAddress)
                      }
              )
          )
      }
  }

@dreamstar-enterprises
Copy link
Author

dreamstar-enterprises commented Dec 23, 2024

No matter how hard I tried, the global WebClient.builder customiser did not work.

So, I ended up using this

  fun createIpv6WebClient(): WebClient {
        return WebClient.builder()
            .clientConnector(
                ReactorClientHttpConnector(
                    HttpClient.create()
                        .resolver(createIpv6Resolver()) // Resolver uses only the host
                        .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
                        .doOnConnected { connection ->
                            val remoteAddress = connection.channel().remoteAddress()
                            logger.debug("Remote Address: {}", remoteAddress)
                        }
                )
            ).build()

    }

And calling it wherever I 'THOUGHT' a webclient was being used...(this did end up working on AWS, without a NAT G/W, on the IPV6 stack, with an IPV6 only egress gateway)

Authorization Token Response

 WebClientReactiveAuthorizationCodeTokenResponseClient().apply {
            setWebClient(webClientGlobalConfig.createIpv6WebClient()

Refresh Token Response

  WebClientReactiveRefreshTokenTokenResponseClient().apply {
                    setWebClient(webClientGlobalConfig.createIpv6WebClient())
                    addParametersConverter { extraParams }
                }

Client Credentials Token Response

   WebClientReactiveClientCredentialsTokenResponseClient().apply {
                    setWebClient(webClientGlobalConfig.createIpv6WebClient())
                    addParametersConverter { extraParams }
                }

RestAPI - ReactiveJwtDecoder


   val jwkSource = JWKSourceBuilder.create(
                CustomJwkSetSource(
                    webClientGlobalConfig.createIpv6WebClient(),
                    jwkSetUri
                )
            )

I maybe missing adding this to TokenResponse for RPInitiatedLogout and BackChannelLogout - but I'm not entirely sure of the Spring Security internals for that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-triage An issue we've not yet triaged type: bug A general bug
Projects
None yet
Development

No branches or pull requests

1 participant