Add envoy proxy examples #277
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for redpanda-labs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
JakeSCahill
force-pushed
the
add-envoy-proxy-example-jake
branch
from
170be8e to
286e262
Compare
Envoy Proxy Lab: - Convert README to AsciiDoc with Doc Detective test annotations - Replace rpk scripts with Python (kafka-python) due to Envoy protocol compatibility - Add code comments explaining architecture and configuration - Add "When to use Envoy vs Shadowing" section with links to combined lab Envoy + Shadowing Lab (new): - Complete DR setup combining Shadowing (data replication) with Envoy (client routing) - Demonstrates shadow link creation, failover workflow, and rpk shadow failover - Uses Python clients with retry logic for robust testing CI/CD: - Add path filters and test steps for both envoy-proxy and envoy-shadowing labs - Add npm test scripts for running Doc Detective tests Docs: - Add symlinks for Antora navigation - Use conditionals for env-site xrefs vs external URLs
JakeSCahill
force-pushed
the
add-envoy-proxy-example-jake
branch
from
286e262 to
4ac76dd
Compare
Redpanda uses Linux AIO (Async I/O). With 6 brokers each needing ~10000 events, they exceed the default fs.aio-max-nr limit of 65536. This was causing broker startup failures in CI, not timeout issues. Added sysctl command to increase aio-max-nr to 1048576 before running envoy tests.
The full 6-broker setup was too resource-intensive for GitHub Actions runners. This adds a CI-specific configuration: - docker-compose.ci.yml: 2 brokers total (1 primary + 1 secondary) - envoy.ci.yaml: Simplified Envoy config for single-broker clusters - README.ci.adoc: CI test spec with adjusted expectations Also includes documentation on using `act` for local CI testing.
Closed
- Remove docker-compose/envoy-proxy directory and all files - Remove envoy-proxy references from GitHub workflow - Remove envoy-proxy test scripts from package.json - Remove envoy-proxy documentation symlinks - Fix disaster-recovery xrefs in envoy-shadowing README (ROOT:manage:disaster-recovery.adoc → ROOT:manage:disaster-recovery/shadowing/index.adoc) - Add explanation of Envoy contrib image requirement in docker-compose.yml - Add Envoy introduction and link in envoy-shadowing README
Member
|
Sorry for the delay @JakeSCahill, I will review this by Monday morning. |
vuldin
approved these changes
Jan 16, 2026
Member
vuldin
left a comment
vuldin
left a comment
There was a problem hiding this comment.
This looks great! I just wanted to point out one limitation I see, which is that the envoy proxy config only focuses on the Kafka port. This is fine for some Kafka clients, but others also would need to use the schema registry. I think it is fine in it's current form (and it has already been useful to customers that don't make use of schemas), but maybe we can plan a future improvement to this demo that adds in schema registry config to Envoy.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add new Envoy + Shadowing lab demonstrating combined DR approach
Envoy + Shadowing Lab (
docker-compose/envoy-shadowing/) - Adapted from #271rpk shadow failoverto enable writesCI/CD
.github/workflows/test-docs.ymltest-envoy-shadowingnpm scriptsTest plan
npm run test-envoy-shadowingpasses (23/23 steps)Previews
https://deploy-preview-277--redpanda-labs-preview.netlify.app/redpanda-labs/docker-compose/envoy-shadowing/