Add a new metrics etcd_server_request_duration_seconds to track e2e duration for each request

[ahrtr]

Fix #21036

Eventually we may need to deprecate etcd_server_range_duration_seconds.

cc @lavacat @fuweid @ivanvc @serathius

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

[codecov]

Codecov Report

❌ Patch coverage is 77.50000% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.42%. Comparing base (88feab3) to head (9cdb1cf).
⚠️ Report is 21 commits behind head on main.

Files with missing lines	Patch %	Lines
server/etcdserver/v3_server.go	77.21%	18 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
server/etcdserver/metrics.go	83.33% <100.00%> (+0.35%)	⬆️
server/etcdserver/v3_server.go	75.53% <77.21%> (+0.57%)	⬆️

... and 25 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #21038      +/-   ##
==========================================
+ Coverage   68.39%   68.42%   +0.03%     
==========================================
  Files         429      429              
  Lines       35213    35291      +78     
==========================================
+ Hits        24083    24149      +66     
- Misses       9721     9739      +18     
+ Partials     1409     1403       -6     

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88feab3...9cdb1cf. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ahrtr]

@tjungblu

[fuweid]

it should have DowngradeCancel + DowngradeEnable

[ahrtr]

DowngradeCancel and DowngradeEnable are already included in DowngradeInfoSet. We don't have to differentiate them.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, fuweid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [ahrtr,fuweid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@ahrtr: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-etcd-coverage-report	9cdb1cf	link	true	/test pull-etcd-coverage-report

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[ahrtr]

thx @fuweid for quick review. @ivanvc @serathius @tjungblu PTAL.

Also I think this metric is important for debugging etcd performance, such as QPS and request latency. Would you be OK with backporting this PR to the stable 3.5 and 3.6 releases? I don’t intend to insist, but I think it would be better to backport.