Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: structured logging [WPB-11544] #739

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: structured logging [WPB-11544] #739

wants to merge 4 commits into from

Conversation

typfel
Copy link
Member

@typfel typfel commented Oct 30, 2024

What's new in this PR

  • Change the logging system to support structured data
  • Add logs to the critical path in proteus and MLS and include relevant contextual information when available.
PR Submission Checklist for internal contributors
  • The PR Title
    • conforms to the style of semantic commits messages¹ supported in Wire's Github Workflow²
    • contains a reference JIRA issue number like SQPIT-764
    • answers the question: If merged, this PR will: ... ³
  1. https://sparkbox.com/foundry/semantic_commit_messages
  2. https://github.com/wireapp/.github#usage
  3. E.g. feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.

@typfel typfel requested a review from a team as a code owner October 30, 2024 15:23
@codecov-commenter
Copy link

codecov-commenter commented Oct 30, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 32.20339% with 40 lines in your changes missing coverage. Please review.

Project coverage is 76.06%. Comparing base (b5f0e7a) to head (a3ba894).

Files with missing lines Patch % Lines
crypto/src/proteus.rs 35.71% 27 Missing ⚠️
crypto-ffi/src/generic.rs 0.00% 11 Missing ⚠️
crypto/src/mls/conversation/encrypt.rs 0.00% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #739      +/-   ##
==========================================
- Coverage   76.19%   76.06%   -0.14%     
==========================================
  Files          99       99              
  Lines       19608    19658      +50     
==========================================
+ Hits        14941    14953      +12     
- Misses       4667     4705      +38
Files with missing lines Coverage Δ
crypto/src/mls/conversation/decrypt.rs 76.81% <100.00%> (+0.08%) ⬆️
crypto/src/mls/conversation/encrypt.rs 59.18% <0.00%> (-0.62%) ⬇️
crypto-ffi/src/generic.rs 0.00% <0.00%> (ø)
crypto/src/proteus.rs 56.42% <35.71%> (-2.42%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5f0e7a...a3ba894. Read the comment docs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 30, 2024 •
edited
Loading

🐰 Bencher Report

Branchfeat/structured-logging-WPB-11544
Testbedubuntu-latest
Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
Commit add f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		11,143,000.00
Commit add f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		764,270.00
Commit add f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		2,916,800.00
Commit add f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		5,116,700.00
Commit add f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		7,850,400.00
Commit add f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		9,759,300.00
Commit add f(number clients)/cs1/mem/1002📈 view plot
🚷 view threshold		979,050,000.00
Commit add f(number clients)/cs1/mem/2📈 view plot
🚷 view threshold		724,670.00
Commit add f(number clients)/cs1/mem/202📈 view plot
🚷 view threshold		78,106,000.00
Commit add f(number clients)/cs1/mem/402📈 view plot
🚷 view threshold		213,300,000.00
Commit add f(number clients)/cs1/mem/602📈 view plot
🚷 view threshold		424,050,000.00
Commit add f(number clients)/cs1/mem/802📈 view plot
🚷 view threshold		664,140,000.00
Commit pending proposals f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		108,930,000.00
Commit pending proposals f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		22,058,000.00
Commit pending proposals f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		39,422,000.00
Commit pending proposals f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		54,248,000.00
Commit pending proposals f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		73,431,000.00
Commit pending proposals f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		87,241,000.00
Commit pending proposals f(pending size)/cs1/mem/1📈 view plot
🚷 view threshold		10,925,000.00
Commit pending proposals f(pending size)/cs1/mem/101📈 view plot
🚷 view threshold		107,250,000.00
Commit pending proposals f(pending size)/cs1/mem/21📈 view plot
🚷 view threshold		28,512,000.00
Commit pending proposals f(pending size)/cs1/mem/41📈 view plot
🚷 view threshold		49,476,000.00
Commit pending proposals f(pending size)/cs1/mem/61📈 view plot
🚷 view threshold		68,096,000.00
Commit pending proposals f(pending size)/cs1/mem/81📈 view plot
🚷 view threshold		87,527,000.00
Commit remove f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		19,391,000.00
Commit remove f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		551,650.00
Commit remove f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		2,204,900.00
Commit remove f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		5,026,200.00
Commit remove f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		9,904,500.00
Commit remove f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		14,219,000.00
Commit remove f(number clients)/cs1/mem/1002📈 view plot
🚷 view threshold		22,092,000.00
Commit remove f(number clients)/cs1/mem/2📈 view plot
🚷 view threshold		129,310,000.00
Commit remove f(number clients)/cs1/mem/202📈 view plot
🚷 view threshold		106,990,000.00
Commit remove f(number clients)/cs1/mem/402📈 view plot
🚷 view threshold		85,970,000.00
Commit remove f(number clients)/cs1/mem/602📈 view plot
🚷 view threshold		65,130,000.00
Commit remove f(number clients)/cs1/mem/802📈 view plot
🚷 view threshold		42,501,000.00
Commit update f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		129,080,000.00
Commit update f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		718,580.00
Commit update f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		26,706,000.00
Commit update f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		52,451,000.00
Commit update f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		79,133,000.00
Commit update f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		103,710,000.00
Count KeyPackage/cs1/mem/1002📈 view plot
🚷 view threshold		3,827,200.00
Count KeyPackage/cs1/mem/2📈 view plot
🚷 view threshold		460,280.00
Count KeyPackage/cs1/mem/202📈 view plot
🚷 view threshold		844,100.00
Count KeyPackage/cs1/mem/402📈 view plot
🚷 view threshold		1,593,300.00
Count KeyPackage/cs1/mem/602📈 view plot
🚷 view threshold		2,323,100.00
Count KeyPackage/cs1/mem/802📈 view plot
🚷 view threshold		3,112,800.00
Create group/cs1/mem📈 view plot
🚷 view threshold		359,110.00
Decrypt f(msg size)/cs1/mem/10📈 view plot
🚷 view threshold		244,840.00
Decrypt f(msg size)/cs1/mem/10010📈 view plot
🚷 view threshold		300,170.00
Decrypt f(msg size)/cs1/mem/2010📈 view plot
🚷 view threshold		253,440.00
Decrypt f(msg size)/cs1/mem/4010📈 view plot
🚷 view threshold		256,380.00
Decrypt f(msg size)/cs1/mem/6010📈 view plot
🚷 view threshold		270,110.00
Decrypt f(msg size)/cs1/mem/8010📈 view plot
🚷 view threshold		287,780.00
Encrypt f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		1,027,600.00
Encrypt f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		279,500.00
Encrypt f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		442,210.00
Encrypt f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		578,630.00
Encrypt f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		721,100.00
Encrypt f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		926,750.00
Encrypt f(msg size)/cs1/mem/10📈 view plot
🚷 view threshold		972,020.00
Encrypt f(msg size)/cs1/mem/10010📈 view plot
🚷 view threshold		1,068,200.00
Encrypt f(msg size)/cs1/mem/2010📈 view plot
🚷 view threshold		978,280.00
Encrypt f(msg size)/cs1/mem/4010📈 view plot
🚷 view threshold		960,210.00
Encrypt f(msg size)/cs1/mem/6010📈 view plot
🚷 view threshold		1,013,200.00
Encrypt f(msg size)/cs1/mem/8010📈 view plot
🚷 view threshold		1,056,800.00
Generate KeyPackage f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		191,720,000.00
Generate KeyPackage f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		553,040.00
Generate KeyPackage f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		22,322,000.00
Generate KeyPackage f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		64,659,000.00
Generate KeyPackage f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		107,080,000.00
Generate KeyPackage f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		150,820,000.00
Join from external commit f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		231,290,000.00
Join from external commit f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		1,447,700.00
Join from external commit f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		47,732,000.00
Join from external commit f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		93,546,000.00
Join from external commit f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		140,340,000.00
Join from external commit f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		185,750,000.00
Join from welcome f(group size)/cs1/mem/1002📈 view plot
🚷 view threshold		107,010,000.00
Join from welcome f(group size)/cs1/mem/2📈 view plot
🚷 view threshold		939,130.00
Join from welcome f(group size)/cs1/mem/202📈 view plot
🚷 view threshold		22,413,000.00
Join from welcome f(group size)/cs1/mem/402📈 view plot
🚷 view threshold		43,678,000.00
Join from welcome f(group size)/cs1/mem/602📈 view plot
🚷 view threshold		65,151,000.00
Join from welcome f(group size)/cs1/mem/802📈 view plot
🚷 view threshold		86,231,000.00
Mls vs Proteus: add/MLS/mem/1📈 view plot
🚷 view threshold		780,830.00
Mls vs Proteus: add/MLS/mem/101📈 view plot
🚷 view threshold		1,854,400.00
Mls vs Proteus: add/MLS/mem/21📈 view plot
🚷 view threshold		1,035,100.00
Mls vs Proteus: add/MLS/mem/41📈 view plot
🚷 view threshold		1,206,500.00
Mls vs Proteus: add/MLS/mem/61📈 view plot
🚷 view threshold		1,449,500.00
Mls vs Proteus: add/MLS/mem/81📈 view plot
🚷 view threshold		1,741,900.00
Mls vs Proteus: add/Proteus/mem/1📈 view plot
🚷 view threshold		450,670.00
Mls vs Proteus: add/Proteus/mem/101📈 view plot
🚷 view threshold		34,564,000.00
Mls vs Proteus: add/Proteus/mem/21📈 view plot
🚷 view threshold		7,340,600.00
Mls vs Proteus: add/Proteus/mem/41📈 view plot
🚷 view threshold		14,127,000.00
Mls vs Proteus: add/Proteus/mem/61📈 view plot
🚷 view threshold		20,906,000.00
Mls vs Proteus: add/Proteus/mem/81📈 view plot
🚷 view threshold		27,721,000.00
Mls vs Proteus: encrypt/MLS/mem/1📈 view plot
🚷 view threshold		276,300.00
Mls vs Proteus: encrypt/MLS/mem/101📈 view plot
🚷 view threshold		375,570.00
Mls vs Proteus: encrypt/MLS/mem/21📈 view plot
🚷 view threshold		293,090.00
Mls vs Proteus: encrypt/MLS/mem/41📈 view plot
🚷 view threshold		321,390.00
Mls vs Proteus: encrypt/MLS/mem/61📈 view plot
🚷 view threshold		344,960.00
Mls vs Proteus: encrypt/MLS/mem/81📈 view plot
🚷 view threshold		351,750.00
Mls vs Proteus: encrypt/Proteus/mem/1📈 view plot
🚷 view threshold		194,120.00
Mls vs Proteus: encrypt/Proteus/mem/101📈 view plot
🚷 view threshold		8,575,800.00
Mls vs Proteus: encrypt/Proteus/mem/21📈 view plot
🚷 view threshold		1,922,200.00
Mls vs Proteus: encrypt/Proteus/mem/41📈 view plot
🚷 view threshold		3,542,000.00
Mls vs Proteus: encrypt/Proteus/mem/61📈 view plot
🚷 view threshold		5,150,200.00
Mls vs Proteus: encrypt/Proteus/mem/81📈 view plot
🚷 view threshold		6,746,000.00
Mls vs Proteus: remove/MLS/mem/1📈 view plot
🚷 view threshold		13,941,000.00
Mls vs Proteus: remove/MLS/mem/101📈 view plot
🚷 view threshold		1,757,900.00
Mls vs Proteus: remove/MLS/mem/21📈 view plot
🚷 view threshold		11,494,000.00
Mls vs Proteus: remove/MLS/mem/41📈 view plot
🚷 view threshold		9,027,300.00
Mls vs Proteus: remove/MLS/mem/61📈 view plot
🚷 view threshold		6,603,000.00
Mls vs Proteus: remove/MLS/mem/81📈 view plot
🚷 view threshold		4,234,800.00
Mls vs Proteus: remove/Proteus/mem/1📈 view plot
🚷 view threshold		100,530.00
Mls vs Proteus: remove/Proteus/mem/101📈 view plot
🚷 view threshold		671,900.00
Mls vs Proteus: remove/Proteus/mem/21📈 view plot
🚷 view threshold		228,780.00
Mls vs Proteus: remove/Proteus/mem/41📈 view plot
🚷 view threshold		344,950.00
Mls vs Proteus: remove/Proteus/mem/61📈 view plot
🚷 view threshold		467,090.00
Mls vs Proteus: remove/Proteus/mem/81📈 view plot
🚷 view threshold		589,790.00
Mls vs Proteus: update/MLS/mem/1📈 view plot
🚷 view threshold		705,470.00
Mls vs Proteus: update/MLS/mem/101📈 view plot
🚷 view threshold		14,363,000.00
Mls vs Proteus: update/MLS/mem/21📈 view plot
🚷 view threshold		3,492,000.00
Mls vs Proteus: update/MLS/mem/41📈 view plot
🚷 view threshold		6,156,600.00
Mls vs Proteus: update/MLS/mem/61📈 view plot
🚷 view threshold		8,680,400.00
Mls vs Proteus: update/MLS/mem/81📈 view plot
🚷 view threshold		11,354,000.00
Mls vs Proteus: update/Proteus/mem/1📈 view plot
🚷 view threshold		459,790.00
Mls vs Proteus: update/Proteus/mem/101📈 view plot
🚷 view threshold		35,321,000.00
Mls vs Proteus: update/Proteus/mem/21📈 view plot
🚷 view threshold		7,489,700.00
Mls vs Proteus: update/Proteus/mem/41📈 view plot
🚷 view threshold		14,409,000.00
Mls vs Proteus: update/Proteus/mem/61📈 view plot
🚷 view threshold		21,333,000.00
Mls vs Proteus: update/Proteus/mem/81📈 view plot
🚷 view threshold		29,063,000.00
🐰 View full continuous benchmarking report in Bencher

@typfel typfel force-pushed the feat/structured-logging-WPB-11544 branch from 4a64729 to df6385d Compare October 30, 2024 16:56
@typfel typfel force-pushed the feat/structured-logging-WPB-11544 branch from df6385d to a3ba894 Compare November 8, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@augustocdias augustocdias augustocdias approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@typfel @codecov-commenter @augustocdias