Thread monitoring for slow requests

[arTG0D]

Currently, 770 java service nodes are monitored using elastic apm java. This greatly shortens the time for locating faults. It's really great. However, when determining an interface with a slow request, I am unable to locate the code position. Therefore, every time, I need to use jstack to export the thread stack for analysis. Can a thread trace for slow requests be added? Similar to glowroot

[JonasKunz]

Are you aware of the inferred spans feature?

It runs a wall clock profiler in the background (so similar to your jstack approach) and tries to automatically generate spans based on the stacktraces it sees.

[jackshirazi]

Additionally, each span already has an associated stack trace (click on the span and it is shown in the "Stack trace" panel or metadata panel depending on UI version and agent with configurable recording time, eg

[arTG0D]

Are you aware of the inferred spans feature?你知道推断跨度的功能吗？

It runs a wall clock profiler in the background (so similar to your jstack approach) and tries to automatically generate spans based on the stacktraces it sees.它在后台运行一个墙时钟分析器（与你的 jstack 方法类似），并尝试根据它看到的堆栈跟踪自动生成跨度。

Elastic stack v8.13.3

apm-agent: elastic-apm-agent-1.52.1.jar

Thank you for your reply! When I set profiling_inferred_spans_enabled=true, an exception error will occur when starting the application, resulting in the startup failure. For details, please refer to: https://discuss.elastic.co/t/apm-java-agent-read-elasticapm-properties-resulting-in-application-startup-failure/374538

[arTG0D]

Additionally, each span already has an associated stack trace (click on the span and it is shown in the "Stack trace" panel or metadata panel depending on UI version and agent with configurable recording time, eg此外，每个跨度已经有一个关联的堆栈跟踪（点击跨度，它会在“堆栈跟踪”面板或元数据面板中显示，具体取决于 UI 版本和代理，具有可配置的记录时间，例如）

Elastic stack v8.13.3

apm-agent: elastic-apm-agent-1.52.1.jar

Thank you for your reply! I used the default Settings span_stack_trace_min_duration and stack_trace_limit in the apm configuration file and displayed them in the Kibana span details. However, when troubleshooting the problem, the stack cannot be seen through the span details because this HTTP request is a transaction. I can only determine that there is a problem with this transaction, but cannot determine the specific code location. As shown in the figure, a large number of redis get and put operations are performed in this transaction. It is found through glowroot slow traces that the blocked operation occurs when the redis get operation is executed.

[jackshirazi]

so if the transaction had the stack trace would that have resolved this?

[arTG0D]

so if the transaction had the stack trace would that have resolved this?那么如果事务有堆栈跟踪，这是否解决了这个问题？

Yes.
It is easier to troubleshoot problems if the thread status at a certain point is displayed in the stack