Add ability to programmatically customize configuration model when otel.experimental.config.file is set

[jack-berg]

The OTEL_EXPERIMENTAL_CONFIG_FILE spec describes a mechanism for the parsed configuration model to be customized:

Implementations MAY provide a mechanism to customize the configuration model parsed from OTEL_EXPERIMENTAL_CONFIG_FILE.

We should add support for this.

[harshitrjpt]

Hi @jack-berg ,
isn't AutoConfigurationCustomizerProvider already providing this capability of programmatically adding configurations?
Eg: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/examples/extension/src/main/java/com/example/javaagent/DemoAutoConfigurationCustomizerProvider.java
Although, currently the SPI implementations are configured first before configuring from the file. But I suppose that order can be changed or not.
What are your thoughts ? Is my interpretation wrong ?

[jack-berg]

AutoConfigurationCustomizerProvider does provide the capability for programmatically customizing configuration, but not for declarative configuration. We could make it so AutoConfigurationCustomizerProvider applies to declarative configuration as well, but that might not be the best idea:

• Implementation probably expect AutoConfigurationCustomizerProvider to be called when env var / system property config is used. It probably doesn't make sense to always run AutoConfigurationCustomizerProvider when declarative config is used.
• AutoConfigurationCustomizerProvider uses the ConfigProperties interface to represent the config. Implementations read from this to make decisions on what / how to customize. ConfigProperties is replaced by StructuredConfigProperties when declarative config is in use, which provides access to data in a significantly different way.

We might need to provide a declarative config alternative to AutoConfigurationCustomizerProvider.

Or we might say that declarative config customization is limited to customizing the config model (i.e. represented in code by OpenTelemetryConfiguration), with a new SPI something like:

public interface DeclarativeConfigurationCustomizer {
  OpenTelemetryConfiguration customizeModel(OpenTelemetryConfiguration);
}

[harshitrjpt]

@jack-berg you are right, AutoConfigurationCustomizerProvider doesn't seem to be a good idea.

However, I investigated and explored more in direction of a new spi like DeclarativeConfigurationCustomizer. These were my observations/ findings (which in IMO achieves more genericness and abstraction)

1. we can reuse the AutoConfigurationCustomizer interface and implement a new DeclarativeConfigurationSdkBuilder. Why? because AutoConfigurationCustomizer already provides the methods to achieve the functions of adding customizers for various components. But we'll need to make AutoConfigurationCustomizer more generic which gets extended by AutoConfiguredOpenTelemetrySdkBuilder and the new DeclarativeConfigurationSdkBuilder in following way.

2. Like you mentioned the only issue in this case is that AutoConfigurationCustomizer uses ConfigProperties in the customizer methods.
   What I propose is that we can go with another super generic interface for properties like Properties extended by the ConfigProperties and StructuredConfigProperties, and use Properties in customizer functions. This will abstract the SDK implementation from the properties type- declarative or no-declarative.

3. Although this will achieve the abstraction, but problem is that all the component configurations- ResourceConfiguration, MeterProviderConfiguration, etc are tightly coupled with ConfigProperties. So we'll need to modify the configurations to use more generic Properties. We'll need to assess how deep these changes would go.

4. Coming back to 1, whatever we do,
   how do we abstract the SDK implementation from choosing which configuration- non-declarative or declarative, to be configured ? or
   we don't and let the SDK implementation consciously decide to use AutoConfiguredOpenTelemetrySdkBuilder or DeclarativeConfigurationSdkBuilder.

( 5. Also, I tried

public interface DeclarativeConfigurationCustomizer {
  OpenTelemetryConfiguration customizeModel(OpenTelemetryConfiguration);
}

in autoconfigure-spi. It will introduce circular dependency between modules 'opentelemetry-java.sdk-extensions.autoconfigure-spi.main' and 'opentelemetry-java.sdk-extensions.autoconfigure.main'.
Also IMO, this will introduce a very specific, different way to customize parse configs even though we already have (generic, if we make it) interface to customize parsed configs.
)

Thoughts, suggestions ?