[OpenTelemetry/Tracing] Interrupted zio effects are not properly traced.

[fredDJSonos]

When using zio.telemetry.opentelemetry.tracing (3.1.10), interrupted zio effect are not recorded correctly.

Depending on the way we record a span, the interruption of the fiber can be totally ignored or just not ideally reported:

• when we use spanScoped, at least an error situation is reported because the implementation is using ZIO.acquireReleaseExit that calls the releaser on interruption. But it would be even better to have a hint of what’s going with otel.status_description == "interrupted". UPDATE: there is something in otel.status_description in the case of an interruption (cause.prettyPrint). It is good enough.
• when we use span or anything else that relies on finalizeSpanUsingEffect, nothing is reported in the span. finalizeSpanUsingEffect is using tapErrorCause, but it looks like tapErrorCause does not catch an interruption.

By the way, the same problem is presumably in v4 since .exit does not intercept an interruption either.

[grouzen]

Hello! Thanks for reporting this. We've never thought about handling the interruption properly. Moreover, the OTEL standard doesn't address it for the time being.

I think the common workaround is to set the status to ERROR and the description to interrupted as you suggested.

when we use span or anything else that relies on finalizeSpanUsingEffect, nothing is reported in the span. finalizeSpanUsingEffect is using tapErrorCause, but it looks like tapErrorCause does not catch an interruption.

finalizeSpanUsingEffect utilises FiberRef.locally under the hood, which calls acquireReleaseWith. I suppose we can call .onInterrupt on it so it can be used the same way as acquireReleaseExit.

[fredDJSonos]

finalizeSpanUsingEffect utilises FiberRef.locally under the hood, which calls acquireReleaseWith. I suppose we can call .onInterrupt on it so it can be used the same way as acquireReleaseExit.

I would suggest something like:

          private def finalizeSpanUsingEffect[R, E, A](
            zio: ZIO[R, E, A],
            ctx: Context,
            statusMapper: StatusMapper[E, A]
          )(implicit trace: Trace): ZIO[R, E, A] =
            ctxStorage
              .locally(ctx)(zio)
              .onExit(exit =>
                exit match {
                  case Exit.Success(a)     => setSuccessStatus(Span.fromContext(ctx), a, statusMapper).unit
                  case Exit.Failure(cause) => setFailureStatus(Span.fromContext(ctx), cause, statusMapper).unit
                }
              )

This avoids doing a special case for interruptions.

[fredDJSonos]

I’ve updated the description a bit. setFailureStatus does a good enough job in the case of an interruption (it logs the original cause in the status_description and that talks about interruptions.

[fredDJSonos]

Hello! Thanks for reporting this. We've never thought about handling the interruption properly. Moreover, the OTEL standard doesn't address it for the time being.

And don't expect it will be addressed anytime soon. Nobody outside the tiny world of zio and friends dare to uncooperatively cancel a remote thread/fiber. And this issue is a perfect illustration of why this is quite dangerous — how can we be sure that everything behaves correctly in the presence of an interruption? What others do is cooperative cancellation and this results with some function calls ending with a particular error situation (like a timeout exception), which is then handled as all other error situations. So yeah, reporting it as an error is the only reasonable choice.

[grouzen]

finalizeSpanUsingEffect utilises FiberRef.locally under the hood, which calls acquireReleaseWith. I suppose we can call .onInterrupt on it so it can be used the same way as acquireReleaseExit.

I would suggest something like:

      private def finalizeSpanUsingEffect[R, E, A](
        zio: ZIO[R, E, A],
        ctx: Context,
        statusMapper: StatusMapper[E, A]
      )(implicit trace: Trace): ZIO[R, E, A] =
        ctxStorage
          .locally(ctx)(zio)
          .onExit(exit =>
            exit match {
              case Exit.Success(a)     => setSuccessStatus(Span.fromContext(ctx), a, statusMapper).unit
              case Exit.Failure(cause) => setFailureStatus(Span.fromContext(ctx), cause, statusMapper).unit
            }
          )

This avoids doing a special case for interruptions.

I like this one! Feel free to create a PR for this!

[dontgitit]

also seeing behavior that I think stems from this issue; I'm using LogFormats to add spanId and traceId to my logs, and they disappear after something as simple as using .timeout:

for {
  _        <- ZIO.logInfo("foo")
  response <- client.request(request)
  _        <- ZIO.logInfo("bar")
  timed    <- response.timeout(duration)
  // log line now missing trace info
  _        <- ZIO.logInfo("baz")
} yield ???

[grouzen]

also seeing behavior that I think stems from this issue; I'm using LogFormats to add spanId and traceId to my logs, and they disappear after something as simple as using .timeout:

for {
_ <- ZIO.logInfo("foo")
response <- client.request(request)
_ <- ZIO.logInfo("bar")
timed <- response.timeout(duration)
// log line now missing trace info
_ <- ZIO.logInfo("baz")
} yield ???

This is weird, indeed. Does .timeout interrupt a fiber? I'm not sure if interruption and cancellation are the same thing in ZIO.

[dontgitit]

hmm, i just tried to reproduce in a small sample app, but it appears to work fine there...

now i'm wondering if the issue stems from something else, like my layer setup/environment... i ran into the issue originally using the otel java agent and OpenTelemetry.global.
i'm used to installing the logging layers in bootstrap, but in my sample app, i noticed that seemed to prevent the tracer from working (kind of) - the otel tracer was still printing everything correctly to the console (using OtlpJsonLoggingSpanExporter), but zio logging wasn't showing spanId/traceId at all. I'm guessing it's creating 2 otel/tracer instances, one in bootstrap and one in run, when i do something like this:

val loggingLayer: URLayer[LogFormats, Unit] = ZLayer {
  Runtime.removeDefaultLoggers >>> myConsoleLogger
}

val tracerLayer = otelSdkLayer >>> (ZioLogging.logFormats ++ OpenTelemetry.tracer("TestTelemetry"))

override val bootstrap: ZLayer[ZIOAppArgs, Any, Any] = {
  tracerLayer >>> loggingLayer
}

override def run = ZIO.serviceWithZIO[Tracer] { tracer =>
    ???
  }
  .provide(
    tracerLayer,
  )

when i removed bootstrap and moved all my logging layers to run, the sample app worked.
is there no way to put logging layers in bootstrap then, when using zio-telemetry?

[dontgitit]

Since I couldn't reproduce, I spent a bit more time digging into what is causing the issue. For some reason, including the Client.default layer seems to trigger this behavior.

Here's a full example to reproduce:

import zio.http.Client
import zio.logging.ConsoleLoggerConfig
import zio.logging.LogFilter
import zio.logging.LogFormat
import zio.telemetry.opentelemetry.core.trace.Tracer
import zio.telemetry.opentelemetry.core.OpenTelemetry as CoreOpenTelemetry
import zio.telemetry.opentelemetry.zio.logging.LogFormats
import zio.telemetry.opentelemetry.zio.logging.ZioLogging
import zio.telemetry.opentelemetry.OpenTelemetry
import zio.Duration
import zio.EnvironmentTag
import zio.Runtime
import zio.UIO
import zio.URLayer
import zio.ZIO
import zio.ZIOApp
import zio.ZLayer

object TestTelemetry extends ZIOApp {
  override type Environment = CoreOpenTelemetry
  override val environmentTag: EnvironmentTag[Environment] = EnvironmentTag[Environment]

  val loggingLayer: URLayer[LogFormats, Unit] = ZLayer {
    for {
      logFormats     <- ZIO.service[LogFormats]
      format          =
        LogFormat.timestamp.fixed(32) |-|
          LogFormat.level |-|
          LogFormat.label("message", LogFormat.quoted(LogFormat.line)) |-|
          logFormats.spanIdLabel |-|
          logFormats.traceIdLabel
      myConsoleLogger =
        zio.logging.consoleLogger(ConsoleLoggerConfig(format.highlight, LogFilter.LogLevelByNameConfig.default))
    } yield Runtime.removeDefaultLoggers >>> myConsoleLogger
  }.flatten

  override val bootstrap = {
    OpenTelemetry.global(logAnnotated = true) >+>
      (ZioLogging.logFormats >>> loggingLayer)
  }

  private def testTimeout(): UIO[Unit] = {
    for {
      _        <- ZIO.logInfo("hello world")
      response <- ZIO.succeed("foo")
      _        <- ZIO.logInfo(s"response is: $response")
      timed    <- ZIO.succeed("bar").timeout(Duration.fromSeconds(5))
      _        <- ZIO.logInfo(s"timed response is: $timed")
    } yield ()
  }

  private def doTest() = for {
    // commenting out the below line makes it work as expected
    client <- ZIO.service[Client]
    tracer <- ZIO.service[Tracer]
    result <- testTimeout() @@ tracer.aspects.span("root_span")
  } yield result

  override def run = doTest()
    .provideSomeAuto(
      // commenting out the below line makes it work as expected
      Client.default,
      OpenTelemetry.tracer("TestTelemetry"),
    )
}

when run with the otel java agent opentelemetry-javaagent-2.22.0.jar, I get the following:

2025-12-08T19:34:15.0337295-06:0 INFO message="hello world" spanId=a200742219478ebd traceId=1bd2686c630ddbf14b08d15984325a73
2025-12-08T19:34:15.038729-06:00 INFO message="response is: foo" spanId=a200742219478ebd traceId=1bd2686c630ddbf14b08d15984325a73
2025-12-08T19:34:15.0522543-06:0 INFO message="timed response is: Some(bar)" spanId=0000000000000000 traceId=00000000000000000000000000000000

if i comment out the lines related to Client (lines 54 and 62), i get the following:

2025-12-08T19:34:48.685164-06:00 INFO message="hello world" spanId=883c5993e2ef97d8 traceId=f7dc05341cc55a79c45b37d94390b164
2025-12-08T19:34:48.7035484-06:0 INFO message="response is: foo" spanId=883c5993e2ef97d8 traceId=f7dc05341cc55a79c45b37d94390b164
2025-12-08T19:34:48.7165672-06:0 INFO message="timed response is: Some(bar)" spanId=883c5993e2ef97d8 traceId=f7dc05341cc55a79c45b37d94390b164

versions:
scala: 3.7.4
zio: 2.1.23
zio-telemetry: 4.0.0-RC9
opentelemetry-api: 1.56.0
zio-http: 3.7.1

if it helps / you have trouble reproducing, i can also generate a minimal sbt build

[dontgitit]

created a full sample repo at https://github.com/dontgitit/zio-telemetry-issue-1069

Some observations:

1. ZIO.succeed("bar").timeout(Duration.fromSeconds(5)) loses context, but something like ZIO.succeed("bar").timeout(Duration.fromSeconds(5)).forkDaemon.flatMap(_.join) does not
2. Removing Client appears to also fix the issue

[dontgitit]

I just saw this related issue... setting -Dotel.instrumentation.zio.enabled=false appears to fix the problem in my sample repo, but doesn't resolve it in my production app...

[grouzen]

also seeing behavior that I think stems from this issue; I'm using LogFormats to add spanId and traceId to my logs, and they disappear after something as simple as using .timeout:
for {
_ <- ZIO.logInfo("foo")
response <- client.request(request)
_ <- ZIO.logInfo("bar")
timed <- response.timeout(duration)
// log line now missing trace info
_ <- ZIO.logInfo("baz")
} yield ???

This is weird, indeed. Does .timeout interrupt a fiber? I'm not sure if interruption and cancellation are the same thing in ZIO.

Yes, it interrupts it according to ZIO docs.

[grouzen]

created a full sample repo at https://github.com/dontgitit/zio-telemetry-issue-1069

Some observations:

1. ZIO.succeed("bar").timeout(Duration.fromSeconds(5)) loses context, but something like ZIO.succeed("bar").timeout(Duration.fromSeconds(5)).forkDaemon.flatMap(_.join) does not
2. Removing Client appears to also fix the issue

Tried to run your repo, can confirm all works as you described. No idea why it behaves this way when using .timeout.
And more mysterious is why it stops losing the context when no Client layer is added.

[grouzen]

I think it can be debugged to find the root cause given the provided repo (thanks for it btw!)