Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add startupProbe in collector pods #1351

Merged

Conversation

lindeskar
Copy link
Contributor

@lindeskar lindeskar commented Sep 18, 2024

Description

Add startupProbe in opentelemetry-collector Pods.

It's basically a copy-paste of stuff from livenessProbe.

Fixes: #1152

--

Background

We were hit by the same issue as #1152; long startup times due to compaction - leading to OTelCol Pods being killed before compaction was completed - leading to another round of the same.

Adding a startupProbe with a higher failureThreshold should be a nice fix.

Sometimes, you have to deal with applications that require additional startup time on their first initialization. In such cases, it can be tricky to set up liveness probe parameters without compromising the fast response to deadlocks that motivated such a probe. The solution is to set up a startup probe with the same command, HTTP or TCP check, with a failureThreshold * periodSeconds long enough to cover the worst case startup time.
From: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-startup-probes

@lindeskar lindeskar marked this pull request as ready for review September 18, 2024 12:45
@lindeskar lindeskar requested a review from a team September 18, 2024 12:45
@lindeskar lindeskar changed the title feat: add startupProbe to collector pods feat: add startupProbe in collector pods Sep 18, 2024
@TylerHelmuth TylerHelmuth merged commit a927332 into open-telemetry:main Sep 19, 2024
3 checks passed
@lindeskar lindeskar deleted the otelcol-startup-probe-1152 branch September 20, 2024 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[opentelemetry-collector] Allow to configure startup probe
2 participants