Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(datastreams): botocore - log warning on kinesis stream metadata not found #11647

Merged
merged 5 commits into from
Dec 17, 2024

Conversation

burnsdy
Copy link
Collaborator

@burnsdy burnsdy commented Dec 9, 2024

For Data Streams Monitoring with AWS Kinesis, the map on the DSM page will break if StreamARN isn't provided to the Kinesis get_records call. This is because dd-trace-py requires the StreamARN when generating a consume checkpoint for an in-edge in the data streams map. If there's no consume checkpoint, the data streams map incorrectly renders as two graphs instead of one connected graph.

However, the Kinesis get_records API lists StreamARN as an optional parameter (see docs). If it isn't provided, dd-trace-py only outputs a debug-level log, and the DSM map is rendered incorrectly. This PR ensures dd-trace-py outputs a warning that is more visible to the developer.

This PR can be closed if the current state of error handling is acceptable, since I don't know if there are restrictions on logging warnings for this repo. The warning helps to debug the broken DSM map, which is time-consuming to debug otherwise.

No testing updates are required since there isn't a unit test file for this specific module.

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@burnsdy burnsdy requested review from a team as code owners December 9, 2024 22:17
Copy link
Contributor

github-actions bot commented Dec 9, 2024

CODEOWNERS have been resolved as:

releasenotes/notes/log-warning-on-kinesis-stream-metadata-not-found-a921cabed5d4397e.yaml  @DataDog/apm-python
ddtrace/internal/datastreams/botocore.py                                @DataDog/apm-core-python

@burnsdy
Copy link
Collaborator Author

burnsdy commented Dec 10, 2024

@emmettbutler Is this good to add to the merge queue?

@emmettbutler
Copy link
Collaborator

@burnsdy looks like it to me!

@burnsdy
Copy link
Collaborator Author

burnsdy commented Dec 11, 2024

/merge

@dd-devflow
Copy link

dd-devflow bot commented Dec 11, 2024

Devflow running: /merge

View all feedbacks in Devflow UI.


2024-12-11 14:45:03 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2024-12-11 18:45:14 UTC ⚠️ MergeQueue: This merge request was unqueued

This merge request was unqueued

@pr-commenter
Copy link

pr-commenter bot commented Dec 11, 2024

Benchmarks

Benchmark execution time: 2024-12-11 16:55:05

Comparing candidate commit 02fa057 in PR branch dylan.burns/dsm-kinesis-error-handling with baseline commit 25e6215 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.

@burnsdy
Copy link
Collaborator Author

burnsdy commented Dec 13, 2024

/merge

@dd-devflow
Copy link

dd-devflow bot commented Dec 13, 2024

Devflow running: /merge

View all feedbacks in Devflow UI.


2024-12-13 17:28:53 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 34m.


2024-12-13 18:00:36 UTCMergeQueue: The checks failed on this merge request

Tests failed on this commit 3977898:

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • Any question, go check the FAQ.

@burnsdy
Copy link
Collaborator Author

burnsdy commented Dec 16, 2024

/merge

@dd-devflow
Copy link

dd-devflow bot commented Dec 16, 2024

Devflow running: /merge

View all feedbacks in Devflow UI.


2024-12-16 20:25:09 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 30m.


2024-12-16 22:25:32 UTCMergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build took longer than expected. The current limit for the base branch 'main' is 120 minutes.

@burnsdy
Copy link
Collaborator Author

burnsdy commented Dec 17, 2024

/merge

@dd-devflow
Copy link

dd-devflow bot commented Dec 17, 2024

Devflow running: /merge

View all feedbacks in Devflow UI.


2024-12-17 16:16:44 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 30m.


2024-12-17 16:49:07 UTC ℹ️ MergeQueue: This merge request was merged

@dd-mergequeue dd-mergequeue bot merged commit 7b77598 into main Dec 17, 2024
270 checks passed
@dd-mergequeue dd-mergequeue bot deleted the dylan.burns/dsm-kinesis-error-handling branch December 17, 2024 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants