-
Notifications
You must be signed in to change notification settings - Fork 399
Description
Tracer Version(s)
2.26.0
Ruby Version(s)
ruby 4.0.1 (2026-01-13 revision e04267a14b) +PRISM [x86_64-linux]
Relevent Library and Version(s)
No response
Bug Report
We upgraded our application to Ruby 4.0.1 a few days ago. Yesterday we encountered an error in production which returned 500 errors to users around 10% of the time. We are running Rails 8.1.2.
After debugging, we found that at least one of the puma threads was blocked, because Thread.current.active_support_execution_state had somehow mutated from a Hash to a Float.
Trace:
/usr/local/bundle/gems/activesupport-8.1.2/lib/active_support/isolated_execution_state.rb:47:in 'ActiveSupport::IsolatedExecutionState.delete': undefined method 'delete' for an instance of Float (NoMethodError)
from /usr/local/bundle/gems/activesupport-8.1.2/lib/active_support/execution_wrapper.rb:68:in 'ActiveSupport::ExecutionWrapper.run!'
from /usr/local/bundle/gems/actionpack-8.1.2/lib/action_dispatch/middleware/executor.rb:14:in 'ActionDispatch::Executor#call'
from /usr/local/bundle/gems/actionpack-8.1.2/lib/action_dispatch/middleware/static.rb:27:in 'ActionDispatch::Static#call'
from /usr/local/bundle/gems/rack-cors-3.0.0/lib/rack/cors.rb:102:in 'Rack::Cors#call'
from /usr/local/bundle/gems/rack-3.2.4/lib/rack/sendfile.rb:131:in 'Rack::Sendfile#call'
from /usr/local/bundle/gems/datadog-2.26.0/lib/datadog/tracing/contrib/rack/middlewares.rb:71:in 'block in Datadog::Tracing::Contrib::Rack::TraceMiddleware#call'
from /usr/local/bundle/gems/datadog-2.26.0/lib/datadog/tracing/contrib/rack/trace_proxy_middleware.rb:17:in 'Datadog::Tracing::Contrib::Rack::TraceProxyMiddleware.call'
from /usr/local/bundle/gems/datadog-2.26.0/lib/datadog/tracing/contrib/rack/middlewares.rb:50:in 'Datadog::Tracing::Contrib::Rack::TraceMiddleware#call'
from /usr/local/bundle/gems/railties-8.1.2/lib/rails/engine.rb:534:in 'Rails::Engine#call'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/configuration.rb:300:in 'Puma::Configuration::ConfigMiddleware#call'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/request.rb:101:in 'block in Puma::Request#handle_request'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/thread_pool.rb:355:in 'Puma::ThreadPool#with_force_shutdown'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/request.rb:100:in 'Puma::Request#handle_request'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/server.rb:503:in 'Puma::Server#process_client'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/server.rb:262:in 'block in Puma::Server#run'
from /usr/local/bundle/gems/puma-7.1.0/lib/puma/thread_pool.rb:182:in 'block in Puma::ThreadPool#spawn_thread'
This only affected one of our Kubernetes pods. We restarted the pod and the problem went away. We have been digging through our dependencies, and we haven't found a clear culprit.
We suspect that this problem might come from Datadog's profiling. It's just a suspicion at the moment, but we thought it would be worth reporting it in case anyone else sees the same problem.
Reproduction Code
No response
Configuration Block
No response
Error Logs
No response
Operating System
No response
How does Datadog help you?
No response