-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(output.kinesis): Log kinesis consumer events #15843
Conversation
e81bd88
to
0591228
Compare
0591228
to
8d52187
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! I just have one comment for now on the code, but could you also open an issue for this PR to resolve? Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mskonovalov for your contribution! I would mark the inner logs of the library with log-level "trace" to be able to selectively enable those as I don't think this is something users need to see all the time!?
switch classification { | ||
case logging.Debug: | ||
t.Logger.Debugf(format, v...) | ||
case logging.Warn: | ||
t.Logger.Warnf(format, v...) | ||
default: | ||
t.Logger.Infof(format, v...) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would use log-level Trace
for all those and add the classification
as prefix to the message. This way we can control if we want to see the inner logs of the library or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TBH I believe it is better to map Warn inside the library to Warn here as well, though Debug and Info I guess is OK to map to Trace.
But it is up to you as a maintainer. So let me know wdyt and I can fix accordingly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I think I didn't get what you mean here :(
If I change it to Trace, then the prefix will be TRACE
, so if I add classification
as a prefix it will look like TRACE DEBUG <message>
. I guess this will be confusing. Or maybe I'm missing something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mskonovalov in normal operations it should not be necessary to see the library-internal log messages! If you map those to e.g. WARN everybody will now see library-internal warnings without a sensible way to ignore those. What I want is that the user CAN enable those log-messages setting the trace level for the plugin in Telegraf if he whishes to see them. You can keep the classification information by prepending the library message like
switch classification {
case logging.Debug:
format = "DEBUG " + format
case logging.Warn:
format = "WARN" + format
case logging.Info:
format = "INFO " + format
}
t.Logger.Tracef(format, v...)
which will produce
<timestamp> T! WARN this is an internal warn message
in case of a warning...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I can make this change.
Although, I'd just mention that I don't fully agree.
If we are using the underlying library, there are not a lot of options how the library can notify us that something is wrong:
- of course, if the issue is critical - it will return the error
- if it is just some informational messages, then it is fine to put it under Trace level
- But what if it is in between: not critical to return error but still something that need to be displayed. E.g. library that writes into some storage can only do it after 3rd retry for every write. The write is successful so it is not returning error, but if we map WARN messages to TRACE level we will never see that something is not working properly.
So I still believe for WARN case it is better to allow to write it to WARN Telegraf log.
But it is up to you to decide.
Though I agree we need to hope that the lib author put only meaningful WARN messages and didn't make it too noisy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well TBH if a library requires logging to propagate important information there is something wrong with the library. A library is an abstraction that should allow the user to use the functionality without knowledge of the internal implementation. On error the library should return an error. If something goes wrong the library should either return some information and let the caller decide on how to continue or return an error and provide means to retry accepting the issue.
So at best the logging serves the purpose of debugging some connection issues or other trouble but this should not be necessary in normal working conditions.
Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip. 📦 Click here to get additional PR build artifactsArtifact URLs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mskonovalov!
@DStrand1 assigning it back to you if you want to have another short look as some of the code changed. Feel free to directly merge the PR! |
Co-authored-by: Mikhail <[email protected]> (cherry picked from commit 336a5e2)
Co-authored-by: Mikhail <[email protected]>
Summary
Currently we have 3 levels of abstraction:
telegraf output kinesis plugin -> kinesis-consumer lib -> aws sdk
Logging is happening only on Telegraf's level. So if there are some warnings or some debug logging on other levels, it cannot be enabled ATM.
In this PR I'm wrapping telegraf logger to comply with interfaces of kinesis-consumer and aws-sdk loggers and pass it when initializing the kinesis client.
For kinesis-consumer there is only a single
Log
function, so I map it to theDEBUG
level.Also enabling logging of retries in aws sdk.
Checklist
Related issues
resolves #15854