Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

needlessly using follow option when retrieving logs of completed job #2264

Open
jrhunger opened this issue Sep 19, 2024 · 2 comments · May be fixed by #2286
Open

needlessly using follow option when retrieving logs of completed job #2264

jrhunger opened this issue Sep 19, 2024 · 2 comments · May be fixed by #2286
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug.
Milestone

Comments

@jrhunger
Copy link
Contributor

What steps did you take and what happened:
Seeing error "illegal base64 data at input byte 5761" in trivy-operator logs.
Looking at the completed job in question, we are able to successfully query and decode the logs:

$ kubectl -n trivy-system logs scan-vulnerabilityreport-d4856fb89-jbzzz  > x
$ wc -c x
5755 x
## note that the illegal input byte 5761 is > the size of this file 5755
$ base64 --decode < x > y
$ head -3 y
{
  "SchemaVersion": 2,
  "CreatedAt": "2024-09-17T09:43:51.613221809Z",

However if we add the -f option to kubectl logs, we get an extra line at the end:

MOo50krocvh7q08U2UkR+LzJgnsPQtfXuwggOKzv/V0EkLKRJH4Peyf9VD/NFLEkT/4x/pWf23/o
82xjGG/1/P9/+L7Ob/x3H/r/8VVPF4/7f7OLpw6dBS/1pUrrVJx/T/zuahnSGIzw5Q/wcrT7KmMw
/4u5IpwoSFKbvOGA
failed to create fsnotify watcher: too many open files

OK, granted this is an issue with our system, however, it also implies that trivy-operator is using the "follow" option when pulling the logs from a completed job, which shouldn't be necessary (no new log entries will be created for a completed job).

What did you expect to happen:
I expect that trivy-operator can retrieve the logs from a completed job even if there is an issue on the system that causes an extra non-base64 line to be added when retrieving logs in Follow mode.

Anything else you would like to add:
Looking in the logs, i see Follow: true in GetLogsByJobAndContainerName
That is only called in pkg/configauditreport/controller/nodecollector.go and pkg/vulnerabilityreport/controller/scanjob.go, in each case inside a function called "processCompleteScanJob" (although there is an intermediate function on the 2nd one)

So if this is always called on Completed jobs, it seems the Follow: true is not necessary.

Environment:

  • Trivy-Operator version (use trivy-operator version): 0.22.0 (but issue is there in current master)
  • Kubernetes version (use kubectl version): 1.27.11 (but not version dependent)
  • OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): Oracle Linux 9.4
@jrhunger jrhunger added the kind/bug Categorizes issue or PR as related to a bug. label Sep 19, 2024
@jrhunger jrhunger linked a pull request Oct 15, 2024 that will close this issue
5 tasks
Copy link

This issue is stale because it has been labeled with inactivity.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Nov 19, 2024
@jrhunger
Copy link
Contributor Author

Should stay active. Pull request pending review.

@afdesk afdesk removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Nov 19, 2024
@simar7 simar7 added this to the v0.24.0 milestone Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants