Skip to content

Conversation

@samikshya-db
Copy link
Collaborator

@samikshya-db samikshya-db commented Dec 29, 2025

Description

  • 4 things that is improved with respect to telemetry :
    • Common object mapper across telemetry use-case (This is already thread safe and is expensive to create, i.e., good tor re-use)
    • MakeflushIntervalMillis config same across both telemetry clients (un-auth and auth)
    • Clear connection param cache when connection is closed : this was a memory leak before
    • Rather than creating a scheduledExecutor for each telemetry client, we share it across a factory.

Testing

  • unit tests

Additional Notes to the Reviewer

  • When the LAST connection to a host is closed, all pending telemetry events for that host are flushed across all prior connections (since they all shared the same TelemetryClient). i.e., If you have 5 connections to host-A, closing connections 1-4 does nothing (just decrements refCount). Only when you close connection 5 (the last one) does the flush occur, sending all accumulated telemetry from all 5 connections.

NO_CHANGELOG=true

@samikshya-db samikshya-db changed the title [PECOBLR-1408] [PECOBLR-1407] Address telemetry code audit comments- part1 [PECOBLR-1408] [PECOBLR-1407] [PECOBLR-1409] Address telemetry code audit comments- part1 Dec 29, 2025
@samikshya-db samikshya-db changed the title [PECOBLR-1408] [PECOBLR-1407] [PECOBLR-1409] Address telemetry code audit comments- part1 [PECOBLR-1408] [PECOBLR-1407] [PECOBLR-1409][PECOBLR-1411] Address telemetry code audit comments- part1 Dec 30, 2025
Copy link
Collaborator

@tejassp-db tejassp-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connection ref count never decrements to zero. Should be fixed.

private InputStream inputStream;
private int chunkReadyTimeoutSeconds =
Integer.parseInt(DatabricksJdbcUrlParams.CHUNK_READY_TIMEOUT_SECONDS.getDefaultValue());
private com.databricks.jdbc.api.internal.IDatabricksConnectionContext connectionContext;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import com.databricks.jdbc.api.internal.IDatabricksConnectionContext into file. Use imports everywhere else as well..

*/
public TelemetryCollector getOrCreateCollector(IDatabricksConnectionContext context) {
String key =
(context != null && context.getConnectionUuid() != null)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for the context or connection uuid to be null? If both are always non-null, then we should enforce that in the code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am skeptical of removing this because of ThreadLocal usage. Check the internal outage here : [Link]

this.executorService = executorService;
this.scheduledExecutorService =
Executors.newSingleThreadScheduledExecutor(createSchedulerThreadFactory());
this.scheduledExecutorService = scheduledExecutorService;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a flush is slow, does it block flush for other statements?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants