ditto-client for java occurs cpu usage 100%

[timyang9527]

my application environmen: java8 springboot ditto-client:2.1.0-m3
my application need manage a lot of iot device ,so, i use a ditto-client pool in my application
I create three class : PoolAutoConfiguration DittoClientPool DittoClientFactory
ditto-client-pool param:
maxIdle: 40
maxTotal: 120
minIdle: 20
initialSize: 20
runtime environmen: docker 2 vm (24cpu 48g memory) 8 pod
when I run my application for some time , There will be 100% CPU usage every ten minutes or so

When I check the jvm, I found this thread ‘ditto-client-scheduled-adaptable-bus-n’

I found this thread in MessagingProviders

` public static MessagingProvider webSocket(final MessagingConfiguration configuration,
final AuthenticationProvider authenticationProvider,
final ExecutorService defaultExecutor) {
final ScheduledExecutorService defaultScheduledExecutor = createScheduledExecutorService(
"adaptable-bus-" + authenticationProvider.getConfiguration().getSessionId());
return WebSocketMessagingProvider.newInstance(configuration, authenticationProvider, defaultExecutor,
defaultScheduledExecutor);
}
public static ScheduledExecutorService createScheduledExecutorService(final String name) {
final int corePoolSize;
if (VersionReader.determineJavaRuntimeVersion() <= 8) {
// for Java <= 8, because of bug https://bugs.openjdk.java.net/browse/JDK-8129861, the corePoolSize must be at least 1:
corePoolSize = 1;
} else {
// bug has been fixed since Java 9, so scale down to 0 threads if the scheduledThreadPool is not needed:
corePoolSize = 0;
}
// ScheduledThreadPool executor is by default unbounded in max size, so start smallest possible core-pool-size:
return Executors.newScheduledThreadPool(corePoolSize,
new DefaultThreadFactory("ditto-client-scheduled-" + name));
}

`

I want to know how to solve this problem？

`public class PoolAutoConfiguration {
private final PoolProperties poolProperties;
private DittoClientPool pool;

@Autowired
public PoolAutoConfiguration(PoolProperties poolProperties) {
    this.poolProperties = poolProperties;
}

@ConditionalOnClass({DittoClientFactory.class})
@Bean
protected DittoClientPool dittoClientPool() {
    DittoClientFactory factory = new DittoClientFactory();
    GenericObjectPoolConfig<DittoClient> poolConfig = new GenericObjectPoolConfig<>();
    poolConfig.setMaxIdle(poolProperties.getMaxIdle());
    poolConfig.setMaxTotal(poolProperties.getMaxTotal());
    poolConfig.setMinIdle(poolProperties.getMinIdle());
    poolConfig.setBlockWhenExhausted(true);
    poolConfig.setTestOnBorrow(true);
    poolConfig.setTestOnReturn(true);
    poolConfig.setTestWhileIdle(true);
    poolConfig.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30);
    poolConfig.setJmxEnabled(false);

    **pool = new DittoClientPool(factory, poolConfig);**

    **initPool(poolProperties.getInitialSize(), poolProperties.getMaxIdle());**
    return pool;
}

/**
 *
 * @param initialSize 
 * @param maxIdle     
 */
private void initPool(int initialSize, int maxIdle) {
    if (initialSize <= 0) {
        return;
    }

    int size = Math.min(initialSize, maxIdle);
    for (int i = 0; i < size; i++) {
        try {
            **pool.addObject();**
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

@PreDestroy
public void destroy() {
    if (pool != null) {
        pool.close();
    }
}

}`

`public class DittoClientPool extends GenericObjectPool {

public DittoClientPool(PooledObjectFactory<DittoClient> factory) {
    super(factory);
}

public DittoClientPool(PooledObjectFactory<DittoClient> factory, GenericObjectPoolConfig<DittoClient> config) {
    super(factory,config);
}

public DittoClientPool(PooledObjectFactory<DittoClient> factory, GenericObjectPoolConfig<DittoClient> config, AbandonedConfig abandonedConfig) {
    super(factory,config,abandonedConfig);
}

}`

`public class DittoClientFactory extends BasePooledObjectFactory {
// @Autowired
// private DittoClientProperties properties;

private static final String PROPERTIES_FILE = "ditto-client-starter-local.properties";
private static final String PROXY_HOST;
private static final String PROXY_PORT;

private static final String DITTO_ENDPOINT_URL;

private static final URL DITTO_TRUSTSTORE_LOCATION;
private static final String DITTO_TRUSTSTORE_PASSWORD;
private static final String DITTO_DUMMY_AUTH_USER;
private static final String DITTO_USERNAME;
private static final String DITTO_PASSWORD;
private static final String DITTO_OAUTH_CLIENT_ID;
private static final String DITTO_OAUTH_CLIENT_SECRET;
private static final Collection<String> DITTO_OAUTH_SCOPES;
private static final String DITTO_OAUTH_TOKEN_ENDPOINT;
private static final String NAMESPACE;
private static final Properties CONFIG;
@Override
public PooledObject<DittoClient> makeObject() throws Exception {
    return super.makeObject();
}

@Override
public void destroyObject(PooledObject<DittoClient> p) throws Exception {
    super.destroyObject(p);
}

@Override
public boolean validateObject(PooledObject<DittoClient> p) {
    return super.validateObject(p);
}

@Override
public void activateObject(PooledObject<DittoClient> p) throws Exception {
    super.activateObject(p);
}

@Override
public void passivateObject(PooledObject<DittoClient> p) throws Exception {
    super.passivateObject(p);
}

@Override
**public DittoClient create() throws Exception {
    return DittoClients.newInstance(createMessagingProvider())
            .connect()
            .toCompletableFuture()
            .join();
}**

@Override
public PooledObject<DittoClient> wrap(DittoClient dittoClient) {
    return new DefaultPooledObject<>(dittoClient);
}


public static MessagingProvider **createMessagingProvider**() {
    final MessagingConfiguration.Builder builder = WebSocketMessagingConfiguration.newBuilder()
            .endpoint(DITTO_ENDPOINT_URL)
            .jsonSchemaVersion(JsonSchemaVersion.V_2)
            .reconnectEnabled(false);

    final ProxyConfiguration proxyConfiguration;
    if (PROXY_HOST != null && !PROXY_HOST.isEmpty()) {
        proxyConfiguration = ProxyConfiguration.newBuilder()
                .proxyHost(PROXY_HOST)
                .proxyPort(Integer.parseInt(PROXY_PORT))
                .build();
        builder.proxyConfiguration(proxyConfiguration);
    } else {
        proxyConfiguration = null;
    }

    if (DITTO_TRUSTSTORE_LOCATION != null) {
        builder.trustStoreConfiguration(TrustStoreConfiguration.newBuilder()
                .location(DITTO_TRUSTSTORE_LOCATION)
                .password(DITTO_TRUSTSTORE_PASSWORD)
                .build());
    }

    final AuthenticationProvider<WebSocket> authenticationProvider;
    if (DITTO_DUMMY_AUTH_USER != null) {
        authenticationProvider =
                AuthenticationProviders.dummy(DummyAuthenticationConfiguration.newBuilder()
                        .dummyUsername(DITTO_DUMMY_AUTH_USER)
                        .build());
    } else if (DITTO_OAUTH_CLIENT_ID != null && !DITTO_OAUTH_CLIENT_ID.isEmpty()) {
        final ClientCredentialsAuthenticationConfiguration.ClientCredentialsAuthenticationConfigurationBuilder
                authenticationConfigurationBuilder =
                ClientCredentialsAuthenticationConfiguration.newBuilder()
                        .clientId(DITTO_OAUTH_CLIENT_ID)
                        .clientSecret(DITTO_OAUTH_CLIENT_SECRET)
                        .scopes(DITTO_OAUTH_SCOPES)
                        .tokenEndpoint(DITTO_OAUTH_TOKEN_ENDPOINT);
        if (proxyConfiguration != null) {
            authenticationConfigurationBuilder.proxyConfiguration(proxyConfiguration);
        }
        authenticationProvider =
                AuthenticationProviders.clientCredentials(authenticationConfigurationBuilder.build());
    } else {
        authenticationProvider =
                AuthenticationProviders.basic(BasicAuthenticationConfiguration.newBuilder()
                        .username(DITTO_USERNAME)
                        .password(DITTO_PASSWORD)
                        .build());
    }

    return MessagingProviders.webSocket(builder.build(), authenticationProvider);
}

static {
    try {
        final Properties config = new Properties();
        if (new File(PROPERTIES_FILE).exists()) {
            config.load(new FileReader(PROPERTIES_FILE));
        } else {
            final InputStream i =
                    Thread.currentThread().getContextClassLoader().getResourceAsStream(PROPERTIES_FILE);
            config.load(i);
            i.close();
        }

        PROXY_HOST = config.getProperty("proxy.host");
        PROXY_PORT = config.getProperty("proxy.port");

        DITTO_ENDPOINT_URL = config.getProperty("ditto.endpoint");

        if (!config.getProperty("ditto.truststore.location").isEmpty()) {
            DITTO_TRUSTSTORE_LOCATION =
                    DittoClientService.class.getResource(config.getProperty("ditto.truststore.location"));
        } else {
            DITTO_TRUSTSTORE_LOCATION = null;
        }
        DITTO_TRUSTSTORE_PASSWORD = config.getProperty("ditto.truststore.password");
        DITTO_DUMMY_AUTH_USER = config.getProperty("ditto.dummy-auth-user");
        DITTO_USERNAME = config.getProperty("ditto.username");
        DITTO_PASSWORD = config.getProperty("ditto.password");
        DITTO_OAUTH_CLIENT_ID = config.getProperty("ditto.oauth.client-id");
        DITTO_OAUTH_CLIENT_SECRET = config.getProperty("ditto.oauth.client-secret");
        DITTO_OAUTH_SCOPES =
                Arrays.stream(config.getProperty("ditto.oauth.scope").split(" ")).collect(Collectors.toSet());
        DITTO_OAUTH_TOKEN_ENDPOINT = config.getProperty("ditto.oauth.token-endpoint");
        NAMESPACE = config.getProperty("ditto.namespace");
        CONFIG = config;
    } catch (final IOException e) {
        throw new IllegalStateException(e);
    }
}

}`

[thjaeckle]

Hi.
2.1.0-M3 is an quite ancient version of the Ditto Java client (and not even an official release, but a milestone).
Do you have the same issues with version 3.0.0?

And do I understand correctly that your devices are connected via the Ditto-Client to the Ditto backend?
How many devices are we talking about? Because the Ditto Java client (and the used WebSocket) are not intended to connect millions of devices - that should be done via a device connectivity layer like e.g. an MQTT broker or Eclipse Hono.