Skip to content

Conversation

@ymuppala
Copy link
Collaborator

Problem Statement

The Venice storage nodes (server-side) lack visibility into the rate of "key not found" events (empty or null responses).

Solution

This PR introduces new server-side metrics to track "key not found" events (empty/null responses) across all read request types (Single Get, Multi-Get, and Compute). These metrics provide visibility into the rate of missing keys at both the per-store and host-aggregate levels, helping identify scenarios where high read traffic results in lookups for non-existent data.

Code changes

  • Added new code behind a config. If so list the config names and their default values in the PR description.
  • Introduced new log lines.
    • Confirmed if logs need to be rate limited to avoid excessive logging.

Concurrency-Specific Checks

Both reviewer and PR author to verify

  • Code has no race conditions or thread safety issues.
  • Proper synchronization mechanisms (e.g., synchronized, RWLock) are used where needed.
  • No blocking calls inside critical sections that could lead to deadlocks or performance degradation.
  • Verified thread-safe collections are used (e.g., ConcurrentHashMap, CopyOnWriteArrayList).
  • Validated proper exception handling in multi-threaded code to avoid silent thread termination.

How was this PR tested?

  • New unit tests added.
  • New integration tests added.
  • Modified or extended existing tests.
  • Verified backward compatibility (if applicable).

Does this PR introduce any user-facing or breaking changes?

  • No. You can skip the rest of this section.
  • Yes. Clearly explain the behavior change and its impact.

@ymuppala ymuppala marked this pull request as draft January 15, 2026 15:48
@ymuppala ymuppala marked this pull request as ready for review January 15, 2026 16:31
Copy link
Contributor

@m-nagarajan m-nagarajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@ymuppala ymuppala enabled auto-merge (squash) January 17, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants