[Enhancement] remove high-cost log location patterns to fix FE latency #69594
Open
yohengyang wants to merge 1 commit intoStarRocks:mainfrom
Open
[Enhancement] remove high-cost log location patterns to fix FE latency #69594yohengyang wants to merge 1 commit intoStarRocks:mainfrom
yohengyang wants to merge 1 commit intoStarRocks:mainfrom
Conversation
…rce resolvers to boost FE throughput
Contributor
Contributor
🌎 Translation Required?✅ All translation files are up to date.
|
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
FE (Frontend) monitoring revealed significant latency spikes in HTTP/JDBC requests, often reaching 10s to 14s. Detailed analysis using Arthas and stack traces identified that the bottleneck was caused by Log4j2's location information retrieval
(%C, %M, %L).Each log entry required a
StackWalker.walk()operation to resolve the class name, method name, and line number. This operation is extremely expensive and caused synchronous blocking in the logging path. This not only delayed query responses but also severely degraded write concurrency, as data load transactions (e.g., Stream Load) were frequently blocked by the audit and profile logging logic.What I'm doing:
Optimized logging performance by removing high-cost stack-walking patterns:
Plaintext Layout: Removed
%C{1}.%M():%Lfrom the log pattern.JSON Layout: Removed
$resolver: source(line, file, method) andset locationInfoEnabled="false".Alternative: Added the logger name resolver (%c) to provide class identification. This uses the pre-cached Logger name instead of walking the stack, providing similar visibility with near-zero overhead.
Fixes #issue

What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: