Skip to content

Commit

Permalink
docs: add tcp vs udp part (#1801)
Browse files Browse the repository at this point in the history
* docs: add tcp vs udp part

* Update architecture.md

* fix: hardcoding poetry version (#1802)

* chore: adding step name (#1803)

Co-authored-by: Lukasz Loboda <[email protected]>
  • Loading branch information
mateuszpierzchala-splunk and uoboda-splunk authored Sep 7, 2022
1 parent 7488473 commit b57ac01
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 6 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/ci-main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,8 @@ jobs:
with:
submodules: false
persist-credentials: false
- run: |
- name: Run tests
run: |
pip3 install poetry
poetry install
mkdir -p test-results || true
Expand Down
22 changes: 19 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,22 @@ _will_ be data loss (think CD-quality (lossless) vs. MP3). Syslog data collecti

## UDP vs. TCP

Paradoxically, UDP for syslog actually ends up being a better choice for resiliency for syslog. For an excellent discussion on this topic
(as well as the "myth" of load balancers for HA),
see [Performant AND Reliable Syslog: UDP is best](https://www.rfaircloth.com/2020/05/21/performant-and-reliable-syslog-udp-is-best/).
For running syslog UDP is recommended over TCP.

The syslogd daemon was originally configured to use UDP for log forwarding to reduce overhead.
While UDP is an unreliable protocol, it's streaming method does not require the overhead of establishing a network session.
This protocol also reduces network load as the network stream with no required receipt verification or window adjustment.
While TCP could seem a better choice because it uses ACKS and there should not be data loss, there are some cases when it's possible:
* The TCP session is closed events published while the system is creating a new session will be lost. (Closed Window Case)
* The remote side is busy and can not ack fast enough events are lost due to local buffer full
* A single ack is lost by the network and the client closes the connection. (local and remote buffer lost)
* The remote server restarts for any reason (local buffer lost)
* The remote server restarts without closing the connection (local buffer plus timeout time lost)
* The client side restarts without closing the connection

Additionally as stated before it causes more overhead on the network.
TCP should be used in case of the syslog event is larger than the maximum size of the UDP packet on your network typically limited to Web Proxy, DLP and IDs type sources.
To decrease drawbacks of TCP you can use TLS over TCP:
* The TLS can continue a session over a broken TCP reducing buffer loss conditions
* The TLS will fill packets for more efficient use of wire
* The TLS will compress in most cases
4 changes: 2 additions & 2 deletions package/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ COPY package/etc/goss.yaml /etc/syslog-ng/goss.yaml

COPY pyproject.toml /
COPY poetry.lock /
RUN pip3 install poetry
RUN pip3 install poetry==1.1.15
RUN poetry export --format requirements.txt | pip3 install --user -r /dev/stdin

COPY package/etc/syslog-ng.conf /etc/syslog-ng/syslog-ng.conf
Expand All @@ -71,4 +71,4 @@ ENV SC4S_CONTAINER_OPTS=--no-caps
ARG VERSION=unknown
RUN echo $VERSION>/etc/syslog-ng/VERSION

ENTRYPOINT ["/entrypoint.sh"]
ENTRYPOINT ["/entrypoint.sh"]

0 comments on commit b57ac01

Please sign in to comment.