diff --git a/docs/configuration.md b/docs/configuration.md index fa58b9d5de..31b506ffd2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -188,7 +188,7 @@ the `example` extension (e.g. `/opt/sc4s/local/context/splunk_metadata.csv`) and file containing a "key" that is referenced in the log path for each data source. These keys are documented in the individual source files in this section, and allow one to override Splunk metadata either in whole or part. The use of this file is best shown by example. Here is the Netscreen "Sourcetype and Index Configuration" table from the Juniper -[source documentation](sources/Juniper/index.md): +[source documentation](sources/vendor/Juniper/index.md): | key | sourcetype | index | notes | |------------------------|---------------------|----------------|---------------| diff --git a/docs/gettingstarted/docker-compose-MacOS.md b/docs/gettingstarted/docker-compose-MacOS.md index c881a4c7f6..e43a8c1cd2 100644 --- a/docs/gettingstarted/docker-compose-MacOS.md +++ b/docs/gettingstarted/docker-compose-MacOS.md @@ -18,37 +18,9 @@ that may have been made to the compose template file below (e.g. suggested mount prior to relaunching via compose. ```yaml -version: "3.7" -services: - sc4s: - image: ghcr.io/splunk/splunk-connect-for-syslog/container2:2 - ports: - - target: 514 - published: 514 - protocol: tcp - - target: 514 - published: 514 - protocol: udp - - target: 601 - published: 601 - protocol: tcp - - target: 6514 - published: 6514 - protocol: tcp - env_file: - - /opt/sc4s/env_file - volumes: - - /opt/sc4s/local:/etc/syslog-ng/conf.d/local:z - - splunk-sc4s-var:/var/lib/syslog-ng -# Uncomment the following line if local disk archiving is desired -# - /opt/sc4s/archive:/var/lib/syslog-ng/archive:z -# Map location of TLS custom TLS -# - /opt/sc4s/tls:/etc/syslog-ng/tls:z - -volumes: - splunk-sc4s-var: +--8<---- "docs/resources/docker-compose.yml" ``` - +* Set `/opt/sc4s` folder as shared in Docker (Settings -> Resources -> File Sharing) * Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in particular the state of the disk buffer. This is a required step. @@ -61,41 +33,15 @@ sudo docker volume create splunk-sc4s-var `/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations (typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configuration page for more info. -* Create the subdirectory `/opt/sc4s/local`. This will be used as a mount point for local overrides and configurations. - - * The empty `local` directory created above will populate with defaults and examples at the first invocation -of SC4S for local configurations and context overrides. _Do not_ change the directory structure of -the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. See the notes below for which files will be preserved on restarts. - - * In the `local/config/` directory there are four subdirectories that allow you to provide support for device types -that are not provided out of the box in SC4S. To get you started, there is an example log path template (`lp-example.conf.tmpl`) -and a filter (`example.conf`) in the `log_paths` and `filters` subdirectories, respectively. These should _not_ be used directly, -but copied as templates for your own log path development. They _will_ get overwritten at each SC4S start. - - * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_metadata.csv`) the changes -will be preserved on a restart. - -* Create the subdirectory `/opt/sc4s/archive`. This will be used as a mount point for local storage of syslog events -(if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" -document for details on the directory structure the archive uses. - -* Create the subdirectory `/opt/sc4s/tls`. This will be used as a mount point for custom TLS certificates -(if the optional mount is uncommented above). - -* IMPORTANT: When creating the directories above, ensure the directories created match the volume mounts specified in the +* IMPORTANT: When creating the directories below, ensure the directories created match the volume mounts specified in the `docker-compose.yml` file (if used). Failure to do this will cause SC4S to abort at startup. -# Configure the SC4S environment +* Create subdirectories `/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls` -SC4S is almost entirely controlled through environment variables, which are read from a file at startup. Create a file named -`/opt/sc4s/env_file` and add the following environment variables and values: +* Create a file named `/opt/sc4s/env_file` and add the following environment variables and values: ```dotenv -SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://splunk.smg.aws:8088 -SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 -#Uncomment the following line if using untrusted SSL certificates -#SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no +--8<--- "docs/resources/env_file" ``` * Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do _not_ configure HEC @@ -108,12 +54,8 @@ deviate from this. * NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to uncomment the last line in the example above. -## Dedicated (Unique) Listening Ports - -For certain source technologies, categorization by message content is impossible due to the lack of a unique "fingerprint" in -the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. -For collection of such sources, we provide a means of dedicating a unique listening port to a specific source. +## Dedicated (Unique) Listening Ports * NOTE: Container networking differs on MacOS compared to that for linux. On Docker Desktop, there is no "host" networking driver, so NAT networking must be used. For this reason, each listening port on the container must be mapped to a listening port on the host. These port mappings are configured in the `docker-compose.yml` file or directly as a runtime option when run out of the CLI. @@ -137,32 +79,9 @@ additional `target` and `published` lines provide for 21 additional technology-s ``` * Restart SC4S using the command in the "Start/Restart SC4S" section below. - -## Modify index destinations for Splunk - -Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers. - -* If changes need to be made to index destinations, navigate to the `/opt/sc4s/local/context` directory to start. -* Edit `splunk_metadata.csv` to review or change the index configuration as required for the data sources utilized in your -environment. The key (1st column) in this file uses the syntax `vendor_product`. Simply replace the index value (the 3rd column) in the -desired row with the index appropriate for your Splunk installation. The "Sources" document details the specific `vendor_product` keys (rows) -in this table that pertain to the individual data source filters that are included with SC4S. -* Other Splunk metadata (e.g. source and sourcetype) can be overridden via this file as well. This is an advanced topic, and further -information is covered in the "Log Path overrides" section of the Configuration document. - -## Configure source filtering by source IP or host name - -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. See documentation -for your vendor/product to determine if specific configuration is required - -## Configure compliance index/metadata overrides - -In some cases, devices that have been properly sourcetyped need to be further categorized by compliance, geography, or other criterion. -The two files `compliance_meta_by_source.conf` and `compliance_meta_by_source.csv` can be used for this purpose. These operate similarly to -the files above, where the `conf` file specifies a filter to uniquely identify the messages that should be overridden, and the `csv` file -lists one or more metadata items that can be overridden based on the filter name. This is an advanced topic, and further information is -covered in the "Override index or metadata based on host, ip, or subnet" section of the Configuration document. +* +For more information about configuration refer to [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations) +and [detailed configuration](../configuration.md). # Start/Restart SC4S @@ -176,12 +95,15 @@ You can use the following command to directly start SC4S if you are not using `d --rm splunk/scs:latest ``` -If you are using `docker-compose`, from the `/opt/sc4s` directory execute: +If you are using `docker-compose`, from the catalog where you created compose file execute: ```bash -docker-compose up --compose-file docker-compose.yml +docker-compose up +``` +Otherwise use `docker-compose` with `-f` flag pointing to the compose file +```bash +docker-compose up -f /path/to/compose/file/docker-compose.yml ``` - # Stop SC4S If the container is run directly from the CLI, simply stop the container using the `docker stop ` command. @@ -189,9 +111,13 @@ If the container is run directly from the CLI, simply stop the container using t If using `docker-compose`, execute: ```bash -docker-compose down --compose-file docker-compose.yml +docker-compose down ``` +or +```bash +docker-compose down -f /path/to/compose/file/docker-compose.yml +``` # Verify Proper Operation SC4S has a number of "preflight" checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng @@ -208,15 +134,12 @@ This should yield an event similar to the following: syslog-ng starting up; version='3.28.1' ``` -when the startup process proceeds normally (without syntax errors). If you do not see this, +When the startup process proceeds normally (without syntax errors). If you do not see this, follow the steps below before proceeding to deeper-level troubleshooting: * Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443). - * Check to see that the proper indexes are created in Splunk, and that the token has access to them. - * Ensure the proper operation of the load balancer if used. - * Lastly, execute the following command to check the sc4s startup process running in the container. ```bash @@ -232,4 +155,5 @@ starting goss starting syslog-ng ``` -If you do not see the output above, proceed to the "Troubleshooting" section for more detailed information. +If you do not see the output above, proceed to the ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md) +and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) sections for more detailed information. diff --git a/docs/gettingstarted/docker-compose.md b/docs/gettingstarted/docker-compose.md new file mode 100644 index 0000000000..d804e10fcb --- /dev/null +++ b/docs/gettingstarted/docker-compose.md @@ -0,0 +1,134 @@ + +# Install Docker Desktop for MacOS + +Refer to [Installation](https://docs.docker.com/engine/install/) + +# SC4S Initial Configuration + +SC4S can be run with `docker-compose` or directly from the CLI with the simple `docker run` command. Both options are outlined below. + +* Create a directory on the server for local configurations and disk buffering. This should be available to all administrators, for example: +`/opt/sc4s/` + +* (Optional for `docker-compose`) Create a docker-compose.yml file in the directory created above, based on the template below: + +* IMPORTANT: Always use the _latest_ compose file (below) with the current release. By default, the latest container is +automatically downloaded at each restart. Therefore, make it a habit to check back here regularly to be sure any changes +that may have been made to the compose template file below (e.g. suggested mount points) are incorporated in production +prior to relaunching via compose. + +``` yaml +--8<---- "docs/resources/docker-compose.yml" +``` + +* Set `/opt/sc4s` folder as shared in Docker (Settings -> Resources -> File Sharing) +* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication +failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in +particular the state of the disk buffer. This is a required step. + +``` +sudo docker volume create splunk-sc4s-var +``` + +* NOTE: Be sure to account for disk space requirements for the docker volume created above. This volume is located in +`/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations +(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configuration page for more info. + +* IMPORTANT: When creating the directories below, ensure the directories created match the volume mounts specified in the +`docker-compose.yml` file (if used). Failure to do this will cause SC4S to abort at startup. + +* Create subdirectories `/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls` + +* Create a file named `/opt/sc4s/env_file` and add the following environment variables and values: + +```dotenv +--8<--- "docs/resources/env_file" +``` + +* Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do _not_ configure HEC +Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this +feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog. + +* The default number of `SC4S_DEST_SPLUNK_HEC_WORKERS` is 10. Consult the community if you feel the number of workers (threads) should +deviate from this. + +* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to +uncomment the last line in the example above. + +For more information about configuration refer to [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations) +and [detailed configuration](../configuration.md). + +# Start/Restart SC4S + +You can use the following command to directly start SC4S if you are not using `docker-compose`. Be sure to map the listening ports +(`-p` arguments) according to your needs: + +```bash +/usr/bin/podman run -p 514:514 -p 514:514/udp -p 6514:6514 -p 5000-5020:5000-5020 -p 5000-5020:5000-5020/udp \ + --env-file=/opt/sc4s/env_file \ + --name SC4S \ + --rm splunk/scs:latest +``` + +If you are using `docker compose`, from the catalog where you created compose file execute: + +```bash +docker compose up +``` +Otherwise use `docker compose` with `-f` flag pointing to the compose file +```bash +docker compose up -f /path/to/compose/file/docker-compose.yml +``` +# Stop SC4S + +If the container is run directly from the CLI, simply stop the container using the `docker stop ` command. + +If using `docker compose`, execute: + +```bash +docker compose down +``` +or + +```bash +docker compose down -f /path/to/compose/file/docker-compose.yml +``` +# Verify Proper Operation + +SC4S has a number of "preflight" checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng +configuration is correct. After this step completes, to verify SC4S is properly communicating with Splunk, +execute the following search in Splunk: + +```ini +index=* sourcetype=sc4s:events "starting up" +``` + +This should yield an event similar to the following: + +```ini +syslog-ng starting up; version='3.28.1' +``` + +When the startup process proceeds normally (without syntax errors). If you do not see this, +follow the steps below before proceeding to deeper-level troubleshooting: + +* Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443). +* Check to see that the proper indexes are created in Splunk, and that the token has access to them. +* Ensure the proper operation of the load balancer if used. +* Lastly, execute the following command to check the sc4s startup process running in the container. + +```bash +docker logs SC4S +``` + +You should see events similar to those below in the output: + +```ini +syslog-ng checking config +sc4s version=v1.36.0 +starting goss +starting syslog-ng +``` + +If you do not see the output above, proceed to the ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md) +and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) sections for more detailed information. diff --git a/docs/gettingstarted/docker-podman-offline.md b/docs/gettingstarted/docker-podman-offline.md new file mode 100644 index 0000000000..ab0f0a2349 --- /dev/null +++ b/docs/gettingstarted/docker-podman-offline.md @@ -0,0 +1,43 @@ +# Offline Container Installation + +Follow these instructions to "stage" SC4S by downloading the container so that it can be loaded "out of band" on a +host machine, such as an airgapped system, without internet connectivity. + +* Download container image "oci_container.tgz" from our [Github Page](https://github.com/splunk/splunk-connect-for-syslog/releases). +The following example downloads v1.12; replace the URL with the latest release or pre-release version as desired. + +``` +sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/v1.12.0/oci_container.tar.gz +``` + +* Distribute the container to the airgapped host machine using an appropriate file transfer utility. +* Execute the following command, using docker or podman as appropriate + +``` + load < oci_container.tar.gz +``` + +* Note the container ID of the resultant load + +``` +Loaded image: docker.pkg.github.com/splunk/splunk-connect-for-syslog/ci:90196f77f7525bc55b3b966b5fa1ce74861c0250 +``` + +* Use the container ID to create a local label +``` + tag docker.pkg.github.com/splunk/splunk-connect-for-syslog/ci:90196f77f7525bc55b3b966b5fa1ce74861c0250 sc4slocal:latest +``` + +* Use this local label `sc4slocal:latest` in the relevant unit or yaml file to launch SC4S (see the runtime options +above) by setting the `SC4S_IMAGE` environment variable in the unit file (example below), or the relevant `image:` tag +if using Docker Compose/Swarm. Using this label will cause the runtime to select the locally loaded image, and will not +attempt to obtain the container image via the internet. + +``` +Environment="SC4S_IMAGE=sc4slocal:latest" +``` +* Remove the entry +``` +ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE +``` +from the relevant unit file when using systemd, as an external connection to pull the container is no longer needed (or available). diff --git a/docs/gettingstarted/docker-systemd-general.md b/docs/gettingstarted/docker-systemd-general.md index 2b9decaff3..05df54e028 100644 --- a/docs/gettingstarted/docker-systemd-general.md +++ b/docs/gettingstarted/docker-systemd-general.md @@ -6,19 +6,7 @@ Refer to relevant installation guides: * [Ubuntu](https://docs.docker.com/install/linux/docker-ce/ubuntu/) * [Debian](https://docs.docker.com/install/linux/docker-ce/debian/) -NOTE: If using a CentOS image provisioned in AWS, IPV4 forwarding is _not_ enabled by default. -This needs to be enabled for container networking to function properly. The following is an example -to set this up; as usual this needs to be vetted with your enterprise security policy: - -```sudo sysctl net.ipv4.ip_forward=1``` - -Then, edit /etc/sysctl.conf, find the text below, and uncomment as shown so that the change made above will survive a -reboot: - -``` -# Uncomment the next line to enable packet forwarding for IPv4 -net.ipv4.ip_forward=1 -``` +NOTE: [READ FIRST (IPv4 forwarding)](./getting-started-runtime-configuration.md#ipv4-forwarding) # Initial Setup @@ -28,49 +16,9 @@ that may have been made to the template unit file below (e.g. suggested mount po to relaunching via systemd. * Create the systemd unit file `/lib/systemd/system/sc4s.service` based on the following template: - +#### Unit file ```ini -[Unit] -Description=SC4S Container -Wants=NetworkManager.service network-online.target docker.service -After=NetworkManager.service network-online.target docker.service -Requires=docker.service - -[Install] -WantedBy=multi-user.target - -[Service] -Environment="SC4S_IMAGE=ghcr.io/splunk/splunk-connect-for-syslog/container2:2" - -# Required mount point for syslog-ng persist data (including disk buffer) -Environment="SC4S_PERSIST_MOUNT=splunk-sc4s-var:/var/lib/syslog-ng" - -# Optional mount point for local overrides and configurations; see notes in docs -Environment="SC4S_LOCAL_MOUNT=/opt/sc4s/local:/etc/syslog-ng/conf.d/local:z" - -# Optional mount point for local disk archive (EWMM output) files -Environment="SC4S_ARCHIVE_MOUNT=/opt/sc4s/archive:/var/lib/syslog-ng/archive:z" - -# Map location of TLS custom TLS -Environment="SC4S_TLS_MOUNT=/opt/sc4s/tls:/etc/syslog-ng/tls:z" - -TimeoutStartSec=0 - -ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE -ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment SC4SHOST=$(hostname -s)" - -ExecStart=/usr/bin/docker run \ - -e "SC4S_CONTAINER_HOST=${SC4SHOST}" \ - -v "$SC4S_PERSIST_MOUNT" \ - -v "$SC4S_LOCAL_MOUNT" \ - -v "$SC4S_ARCHIVE_MOUNT" \ - -v "$SC4S_TLS_MOUNT" \ - --env-file=/opt/sc4s/env_file \ - --network host \ - --name SC4S \ - --rm $SC4S_IMAGE - -Restart=on-abnormal +--8<--- "docs/resources/docker/sc4s.service" ``` * Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication @@ -85,41 +33,12 @@ sudo docker volume create splunk-sc4s-var `/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations (typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configuration page for more info. -* Create the subdirectory `/opt/sc4s/local`. This will be used as a mount point for local overrides and configurations. - - * The empty `local` directory created above will populate with defaults and examples at the first invocation -of SC4S for local configurations and context overrides. _Do not_ change the directory structure of -the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. See the notes below for which files will be preserved on restarts. - - * In the `local/config/` directory there are four subdirectories that allow you to provide support for device types -that are not provided out of the box in SC4S. To get you started, there is an example log path template (`lp-example.conf.tmpl`) -and a filter (`example.conf`) in the `log_paths` and `filters` subdirectories, respectively. These should _not_ be used directly, -but copied as templates for your own log path development. They _will_ get overwritten at each SC4S start. - - * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_metadata.csv`) the changes -will be preserved on a restart. - -* Create the subdirectory `/opt/sc4s/archive`. This will be used as a mount point for local storage of syslog events -(if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" -document for details on the directory structure the archive uses. - -* Create the subdirectory `/opt/sc4s/tls`. This will be used as a mount point for custom TLS certificates -(if the optional mount is uncommented above). +* Create subdirectories `/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls` -* IMPORTANT: When creating the directories above, ensure the directories created match the volume mounts specified in the -unit file above. Failure to do this will cause SC4S to abort at startup. - -# Configure the sc4s environment - -SC4S is almost entirely controlled through environment variables, which are read from a file at startup. Create a file named -`/opt/sc4s/env_file` and add the following environment variables and values: +* Create a file named `/opt/sc4s/env_file` and add the following environment variables and values: ```dotenv -SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://splunk.smg.aws:8088 -SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 -#Uncomment the following line if using untrusted SSL certificates -#SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no +--8<--- "docs/resources/env_file" ``` * Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do _not_ configure HEC @@ -132,58 +51,17 @@ deviate from this. * NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to uncomment the last line in the example above. -## Dedicated (Unique) Listening Ports - -For certain source technologies, categorization by message content is impossible due to the lack of a unique "fingerprint" in -the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. -For collection of such sources, we provide a means of dedicating a unique listening port to a specific source. - -Follow this step to configure unique ports for one or more sources: - -* Modify the `/opt/sc4s/env_file` file to include the port-specific environment variable(s). Refer to the "Sources" -documentation to identify the specific environment variables that are mapped to each data source vendor/technology. - -## Modify index destinations for Splunk - -Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers. - -* If changes need to be made to index destinations, navigate to the `/opt/sc4s/local/context` directory to start. -* Edit `splunk_metadata.csv` to review or change the index configuration as required for the data sources utilized in your -environment. The key (1st column) in this file uses the syntax `vendor_product`. Simply replace the index value (the 3rd column) in the -desired row with the index appropriate for your Splunk installation. The "Sources" document details the specific `vendor_product` keys (rows) -in this table that pertain to the individual data source filters that are included with SC4S. -* Other Splunk metadata (e.g. source and sourcetype) can be overridden via this file as well. This is an advanced topic, and further -information is covered in the "Log Path overrides" section of the Configuration document. - -## Configure source filtering by source IP or host name - -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. See documentation -for your vendor/product to determine if specific configuration is required - -## Configure compliance index/metadata overrides +For more information about configuration refer to [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations) +and [detailed configuration](../configuration.md). -In some cases, devices that have been properly sourcetyped need to be further categorized by compliance, geography, or other criterion. -The two files `compliance_meta_by_source.conf` and `compliance_meta_by_source.csv` can be used for this purpose. These operate similarly to -the files above, where the `conf` file specifies a filter to uniquely identify the messages that should be overridden, and the `csv` file -lists one or more metadata items that can be overridden based on the filter name. This is an advanced topic, and further information is -covered in the "Override index or metadata based on host, ip, or subnet" section of the Configuration document. - -## Configure SC4S for systemd and start SC4S +# Configure SC4S for systemd and start SC4S ```bash sudo systemctl daemon-reload sudo systemctl enable sc4s sudo systemctl start sc4s ``` - -# Start SC4S - -```bash -sudo systemctl start sc4s -``` - -# Restart SC4S +## Restart SC4S ```bash sudo systemctl restart sc4s @@ -199,7 +77,7 @@ sudo systemctl enable sc4s sudo systemctl start sc4s ``` -# Stop SC4S +## Stop SC4S ```bash sudo systemctl stop sc4s @@ -221,15 +99,12 @@ This should yield an event similar to the following: syslog-ng starting up; version='3.28.1' ``` -when the startup process proceeds normally (without syntax errors). If you do not see this, +When the startup process proceeds normally (without syntax errors). If you do not see this, follow the steps below before proceeding to deeper-level troubleshooting: * Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443). - * Check to see that the proper indexes are created in Splunk, and that the token has access to them. - * Ensure the proper operation of the load balancer if used. - * Lastly, execute the following command to check the sc4s startup process running in the container. ```bash @@ -245,4 +120,5 @@ starting goss starting syslog-ng ``` -If you do not see the output above, proceed to the "Troubleshooting" section for more detailed information. +If you do not see the output above, proceed to the ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md) +and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) sections for more detailed information. diff --git a/docs/gettingstarted/getting-started-runtime-configuration.md b/docs/gettingstarted/getting-started-runtime-configuration.md new file mode 100644 index 0000000000..5cc71aa4cb --- /dev/null +++ b/docs/gettingstarted/getting-started-runtime-configuration.md @@ -0,0 +1,125 @@ + +# Implement a Container Runtime and SC4S + +## Prerequisites + +* Linux host with Docker (CE 19.x or greater) or Podman enabled, depending on runtime choice (below). +* A network load balancer (NLB) configured for round-robin. Note: Special consideration may be required when more advanced products are used. +The optimal configuration of the load balancer will round-robin each http POST request (not each connection). +* The host linux OS receive buffer size should be tuned to match the sc4s default to avoid dropping events (packets) at the network level. +The default receive buffer for sc4s is set to 16 MB for UDP traffic, which should be OK for most environments. To set the host OS kernel to +match this, edit `/etc/sysctl.conf` using the following whole-byte values corresponding to 16 MB: + +```bash +net.core.rmem_default = 17039360 +net.core.rmem_max = 17039360 +``` +and apply to the kernel: +```bash +sysctl -p +``` +* Ensure the kernel is not dropping packets by periodically monitoring the buffer with the command +`netstat -su | grep "receive errors"`. +* NOTE: Failure to account for high-volume traffic (especially UDP) by tuning the kernel will result in message loss, which can be _very_ +unpredictable and difficult to detect. See this helpful discussion in the syslog-ng +[Professional Edition](https://www.syslog-ng.com/technical-documents/doc/syslog-ng-premium-edition/7.0.10/collecting-log-messages-from-udp-sources) +documentation regarding tuning syslog-ng in particular (via the [SC4S_SOURCE_*_SO_RCVBUFF](../configuration.md#syslog-source-configuration) +environment variable in sc4s) as well as overall host kernel tuning. The default values for receive kernel buffers in most distros is 2 MB, +which has proven inadequate for many. + +## IPv4 Forwarding + +In many distributions (e.g. CentOS provisioned in AWS), IPV4 forwarding is _not_ enabled by default. +This needs to be enabled for container networking to function properly. The following is an example +to check and set this up; as usual this needs to be vetted with your enterprise security policy: + +To check: +```sudo sysctl net.ipv4.ip_forward``` +To set: +```sudo sysctl net.ipv4.ip_forward=1``` + +To ensure the change survives a reboot: + +* sysctl settings are defined through files in ```/usr/lib/sysctl.d/```, ```/run/sysctl.d/```, and ```/etc/sysctl.d/```. +* To override only specific settings, you can either add a file with a lexically later name in ```/etc/sysctl.d/``` and put following setting there: +``` +net.ipv4.ip_forward=1 +``` +* or find this specific setting in one of existing configuration files (mentioned above) and set value to ```1```. + +``` +net.ipv4.ip_forward=1 +``` +## Select a Container Runtime and SC4S Configuration + +| Container Runtime and Orchestration | Operating Systems | +|-------------------------------------------------------------------|-----------------------------------------------------------------| +| [MicroK8s](k8s-microk8s.md) | Ubuntu with Microk8s | +| [Podman 1.7 & 1.9 + systemd](podman-systemd-general.md) | RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS | +| [Docker CE 19 (and greater) + systemd](docker-systemd-general.md) | RHEL or CentOS >7.7 (best option), Debian or Ubuntu 18.04LTS | +| [Docker Desktop + Compose](docker-compose-MacOS.md) | MacOS | +| [Docker Desktop + Compose](docker-compose.md) | RHEL or CentOS 8.1 & 8.2 (best option) | +| [Bring your own Environment](byoe-rhel8.md) | RHEL or CentOS 8.1 & 8.2 (best option) | +| [Offline Container Installation](docker-podman-offline.md) | RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS | + +### Docker and Podman basic configurations +* To run properly sc4s you need to create directories:`/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls` +* `/opt/sc4s/local` will be used as a mount point for local overrides and configurations. +The empty `local` directory created above will populate with defaults and examples at the first invocation +of SC4S for local configurations and context overrides. _Do not_ change the directory structure of +the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout +to read the local configurations properly. See the notes below for which files will be preserved on restarts. +In the `local/config/` directory there are four subdirectories that allow you to provide support for device types +that are not provided out of the box in SC4S. To get you started, there is an example log path template (`lp-example.conf.tmpl`) +and a filter (`example.conf`) in the `log_paths` and `filters` subdirectories, respectively. These should _not_ be used directly, +but copied as templates for your own log path development. They _will_ get overwritten at each SC4S start. +In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_metadata.csv`) the changes +will be preserved on a restart. + +* `/opt/sc4s/archive` will be used as a mount point for local storage of syslog events +(if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" +document for details on the directory structure the archive uses. + +* `/opt/sc4s/tls` will be used as a mount point for custom TLS certificates +(if the optional mount is uncommented above). + +* IMPORTANT: When creating the directories above, ensure the directories created match the volume mounts specified in the +sc4s.service [unit file](podman-systemd-general.md#unit-file). Failure to do this will cause SC4S to abort at startup. + + +#### Dedicated (Unique) Listening Ports + +For certain source technologies, categorization by message content is impossible due to the lack of a unique "fingerprint" in +the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. +For collection of such sources, we provide a means of dedicating a unique listening port to a specific source. + +Follow this step to configure unique ports for one or more sources: + +* Modify the `/opt/sc4s/env_file` file to include the port-specific environment variable(s). Refer to the ["Sources"](../sources/index.md) +documentation to identify the specific environment variables that are mapped to each data source vendor/technology. + +#### Modify index destinations for Splunk + +Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers. + +* If changes need to be made to index destinations, navigate to the `/opt/sc4s/local/context` directory to start. +* Edit `splunk_metadata.csv` to review or change the index configuration as required for the data sources utilized in your +environment. The key (1st column) in this file uses the syntax `vendor_product`. Simply replace the index value (the 3rd column) in the +desired row with the index appropriate for your Splunk installation. The "Sources" document details the specific `vendor_product` keys (rows) +in this table that pertain to the individual data source filters that are included with SC4S. +* Other Splunk metadata (e.g. source and sourcetype) can be overridden via this file as well. This is an advanced topic, and further +information is covered in the "Log Path overrides" section of the Configuration document. + +#### Configure source filtering by source IP or host name + +Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps +apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. See documentation +for your vendor/product to determine if specific configuration is required + +#### Configure compliance index/metadata overrides + +In some cases, devices that have been properly sourcetyped need to be further categorized by compliance, geography, or other criterion. +The two files `compliance_meta_by_source.conf` and `compliance_meta_by_source.csv` can be used for this purpose. These operate similarly to +the files above, where the `conf` file specifies a filter to uniquely identify the messages that should be overridden, and the `csv` file +lists one or more metadata items that can be overridden based on the filter name. This is an advanced topic, and further information is +covered in the "Override index or metadata based on host, ip, or subnet" section of the Configuration document. diff --git a/docs/gettingstarted/getting-started-splunk-setup.md b/docs/gettingstarted/getting-started-splunk-setup.md new file mode 100644 index 0000000000..3a9dcfe7a1 --- /dev/null +++ b/docs/gettingstarted/getting-started-splunk-setup.md @@ -0,0 +1,55 @@ +# Splunk setup +## Create Indexes + +SC4S is pre-configured to map each sourcetype to a typical index. For new installations, it is best practice to create them in Splunk when +using the SC4S defaults. SC4S can be easily customized to use different indexes if desired. + +* email +* epav +* epintel +* infraops +* netauth +* netdlp +* netdns +* netfw +* netids +* netlb +* netops +* netwaf +* netproxy +* netipam +* oswin +* oswinsec +* osnix +* print +* em_metrics (Optional opt-in for SC4S operational metrics; ensure this is created as a metrics index) + +#### Install Related Splunk Apps + +Install the following: + +* [IT Essentials Work](https://splunkbase.splunk.com/app/5403/) + +#### Configure the Splunk HTTP Event Collector + +- Set up the Splunk HTTP Event Collector with the HEC endpoints behind a load balancer (VIP) configured for https round robin *WITHOUT* sticky +session. Alternatively, a list of HEC endpoint URLs can be configured in SC4S (native syslog-ng load balancing) if no load balancer is in +place. In most scenarios the recommendation is to use an external load balancer, as that makes longer term +maintenance simpler by eliminating the need to manually keep the list of HEC URLs specified in sc4s current. However, if a LB is not +available, native load balancing can be used with 10 or fewer Indexers where HEC is used exclusively for syslog. + + In either case, it is _strongly_ recommended that SC4S traffic be sent to HEC endpoints configured directly on the indexers rather than +an intermediate tier of HWFs. +- Create a HEC token that will be used by SC4S and ensure the token has access to place events in main, em_metrics, and all indexes used as +event destinations. + +* NOTE: It is recommended that the "Selected Indexes" on the token configuration page be left blank so that the token has access to +_all_ indexes, including the `lastChanceIndex`. If this list is populated, extreme care must be taken to keep it up to date, as an attempt to +send data to an index not in this list will result in a `400` error from the HEC endpoint. Furthermore, the `lastChanceIndex` will _not_ be +consulted in the event the index specified in the event is not configured on Splunk. Keep in mind just _one_ bad message will "taint" the +whole batch (by default 1000 events) and prevent the entire batch from being sent to Splunk. +* In case you are not using TLS on SC4S- turn off SSL on global settings for HEC in Splunk. +- Refer to [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_managed_Splunk_Cloud) +or [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q) for specific HEC configuration instructions based on your +Splunk type. + diff --git a/docs/gettingstarted/index.md b/docs/gettingstarted/index.md index d1872b8e97..208d95f13a 100644 --- a/docs/gettingstarted/index.md +++ b/docs/gettingstarted/index.md @@ -15,170 +15,19 @@ transmission of events between computer systems over UDP, TCP, or TLS. The proto overhead on the sender favoring performance over reliability. This fundamental choice means any instability or resource constraint will cause data to be lost in transmission. -* When practical and cost effective (considering the importance of completeness as a requirement), place the sc4s +* When practical and cost-effective (considering the importance of completeness as a requirement), place the sc4s instance in the same VLAN as the source device. * Avoid crossing a Wireless network, WAN, Firewall, Load Balancer, or inline IDS. * When High Availability of a single instance of SC4S is required, implement multi node clustering of the container environment. * Avoid TCP except where the source is unable to contain the event to a single UDP packet. -* Avoid TLS except where the event may cross a untrusted network. +* Avoid TLS except where the event may cross untrusted network. * Plan for [appropriately sized hardware](../performance.md) ## Implementation -### Splunk Setup +### [Splunk Setup](getting-started-splunk-setup.md) -#### Create Indexes - -SC4S is pre-configured to map each sourcetype to a typical index. For new installations, it is best practice to create them in Splunk when -using the SC4S defaults. SC4S can be easily customized to use different indexes if desired. - -* email -* epav -* epintel -* infraops -* netauth -* netdlp -* netdns -* netfw -* netids -* netlb -* netops -* netwaf -* netproxy -* netipam -* oswin -* oswinsec -* osnix -* print -* em_metrics (Optional opt-in for SC4S operational metrics; ensure this is created as a metrics index) - -#### Install Related Splunk Apps - -Install the following: - -* [IT Essentials Work](https://splunkbase.splunk.com/app/5403/) - -#### Configure the Splunk HTTP Event Collector - -- Set up the Splunk HTTP Event Collector with the HEC endpoints behind a load balancer (VIP) configured for https round robin *WITHOUT* sticky -session. Alternatively, a list of HEC endpoint URLs can be configured in SC4S (native syslog-ng load balancing) if no load balancer is in -place. In most scenarios the recommendation is to use an external load balancer, as that makes longer term -maintenance simpler by eliminating the need to manually keep the list of HEC URLs specified in sc4s current. However, if a LB is not -available, native load balancing can be used with 10 or fewer Indexers where HEC is used exclusively for syslog. - - In either case, it is _strongly_ recommended that SC4S traffic be sent to HEC endpoints configured directly on the indexers rather than -an intermediate tier of HWFs. -- Create a HEC token that will be used by SC4S and ensure the token has access to place events in main, em_metrics, and all indexes used as -event destinations. - -* NOTE: It is recommended that the "Selected Indexes" on the token configuration page be left blank so that the token has access to -_all_ indexes, including the `lastChanceIndex`. If this list is populated, extreme care must be taken to keep it up to date, as an attempt to -send data to an index not in this list will result in a `400` error from the HEC endpoint. Furthermore, the `lastChanceIndex` will _not_ be -consulted in the event the index specified in the event is not configured on Splunk. Keep in mind just _one_ bad message will "taint" the -whole batch (by default 1000 events) and prevent the entire batch from being sent to Splunk. - -- Refer to [Splunk Cloud](http://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector#Configure_HTTP_Event_Collector_on_managed_Splunk_Cloud) -or [Splunk Enterprise](http://dev.splunk.com/view/event-collector/SP-CAAAE6Q) for specific HEC configuration instructions based on your -Splunk type. - -### Implement a Container Runtime and SC4S - -#### Prerequisites - -* Linux host with Docker (CE 19.x or greater with Docker Swarm) or Podman enabled, depending on runtime choice (below). -* A network load balancer (NLB) configured for round robin. Note: Special consideration may be required when more advanced products are used. -The optimal configuration of the load balancer will round robin each http POST request (not each connection). -* The host linux OS receive buffer size should be tuned to match the sc4s default to avoid dropping events (packets) at the network level. -The default receive buffer for sc4s is set to 16 MB for UDP traffic, which should be OK for most environments. To set the host OS kernel to -match this, edit `/etc/sysctl.conf` using the following whole-byte values corresponding to 16 MB: - -```bash -net.core.rmem_default = 17039360 -net.core.rmem_max = 17039360 -``` -and apply to the kernel: -```bash -sysctl -p -``` -* Ensure the kernel is not dropping packets by periodically monitoring the buffer with the command -`netstat -su | grep "receive errors"`. -* NOTE: Failure to account for high-volume traffic (especially UDP) by tuning the kernel will result in message loss, which can be _very_ -unpredictable and difficult to detect. See this helpful discussion in the syslog-ng -[Professional Edition](https://www.syslog-ng.com/technical-documents/doc/syslog-ng-premium-edition/7.0.10/collecting-log-messages-from-udp-sources) -documentation regarding tuning syslog-ng in particular (via the [SC4S_SOURCE_*_SO_RCVBUFF](../configuration.md#syslog-source-configuration) -environment variable in sc4s) as well as overall host kernel tuning. The default values for receive kernel buffers in most distros is 2 MB, -which has proven inadequate for many. - -#### IPv4 Forwarding - -In many distributions (e.g. CentOS provisioned in AWS), IPV4 forwarding is _not_ enabled by default. -This needs to be enabled for container networking to function properly. The following is an example -to set this up; as usual this needs to be vetted with your enterprise security policy: - -```sudo sysctl net.ipv4.ip_forward=1``` - -To ensure the change survives a reboot edit /etc/sysctl.conf, find (or add) the text below, and uncomment as shown: - -``` -# Uncomment the next line to enable packet forwarding for IPv4 -net.ipv4.ip_forward=1 -``` - -#### Select a Container Runtime and SC4S Configuration - -| Container Runtime and Orchestration | Operating Systems | -|-----------------------------|-------| -| [MicroK8s](k8s-microk8s.md) | Ubuntu with Microk8s | -| [Podman 1.7 & 1.9 + systemd](podman-systemd-general.md) | RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS | -| [Docker CE 18 & 19 + systemd](docker-systemd-general.md) | RHEL or CentOS >7.7 (best option), Debian or Ubuntu 18.04LTS | -| [Docker CE 18 & 19 + Swarm](docker-swarm-general.md) | CentOS >7.7 (best option), Debian or Ubuntu 18.04LTS | -| [Docker CE 18 & 19 + Swarm](docker-swarm-rhel7.md) | RHEL 7.7 Deprecated | -| [Docker Desktop + Compose](docker-compose-MacOS.md) | MacOS | -| [Bring your own Environment](byoe-rhel8.md) | RHEL or CentOS 8.1 & 8.2 (best option) | - -### Offline Container Installation - -Follow these instructions to "stage" SC4S by downloading the container so that it can be loaded "out of band" on a -host machine, such as an airgapped system, without internet connectivity. - -* Download container image "oci_container.tgz" from our [Github Page](https://github.com/splunk/splunk-connect-for-syslog/releases). -The following example downloads v1.12; replace the URL with the latest release or pre-release version as desired. - -``` -sudo wget https://github.com/splunk/splunk-connect-for-syslog/releases/download/v1.12.0/oci_container.tar.gz -``` - -* Distribute the container to the airgapped host machine using an appropriate file transfer utility. -* Execute the following command, using docker or podman as appropriate - -``` - load < oci_container.tar.gz -``` - -* Note the container ID of the resultant load - -``` -Loaded image: docker.pkg.github.com/splunk/splunk-connect-for-syslog/ci:90196f77f7525bc55b3b966b5fa1ce74861c0250 -``` - -* Use the container ID to create a local label -``` - tag docker.pkg.github.com/splunk/splunk-connect-for-syslog/ci:90196f77f7525bc55b3b966b5fa1ce74861c0250 sc4slocal:latest -``` - -* Use this local label `sc4slocal:latest` in the relevant unit or yaml file to launch SC4S (see the runtime options -above) by setting the `SC4S_IMAGE` environment variable in the unit file (example below), or the relevant `image:` tag -if using Docker Compose/Swarm. Using this label will cause the runtime to select the locally loaded image, and will not -attempt to obtain the container image via the internet. - -``` -Environment="SC4S_IMAGE=sc4slocal:latest" -``` -* Remove the entry -``` -ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE -``` -from the relevant unit file when using systemd, as an external connection to pull the container is no longer needed (or available). +### [Runtime configuration](getting-started-runtime-configuration.md) diff --git a/docs/gettingstarted/k8s-microk8s.md b/docs/gettingstarted/k8s-microk8s.md index 82228c1360..7edc36dd26 100644 --- a/docs/gettingstarted/k8s-microk8s.md +++ b/docs/gettingstarted/k8s-microk8s.md @@ -1,6 +1,5 @@ # Install MicroK8s - The SC4S deployment model with Microk8s uses specific features of this distribution of k8s. While this may be reproducible with other distributions such an undertaking requires more advanced awareness and responsibility for the administrator. @@ -12,7 +11,6 @@ This configuration requires as least 2 IP addressed one for host and one for the We suggest allocation of 3 ip addresses for the host and 5-10 addresses for later use # FAQ - Question: How is this deployment model supported? Answer: Similar to other deployment methods, Splunk supports the container itself and the procedural guidance for implementation but does not directly support or otherwise provide resolutions for issues within the runtime environment. @@ -28,7 +26,7 @@ on a deployment-specific basis. ```bash #we need to have a normal install of kubectl because of operator scripts -sudo snap install kubectl --classic +sudo snap install microk8s --classic --channel=1.24 # Basic setup of k8s sudo usermod -a -G microk8s $USER sudo chown -f -R $USER ~/.kube @@ -39,9 +37,15 @@ microk8s status --wait-ready #Into the cluster if your plan to enable clustering this IP should not be assigned to the host (floats) #If you do not plan to cluster then this IP may be the same IP as the host #Note2: a single IP in cidr format is x.x.x.x/32 use CIDR or range syntax -microk8s enable dns metallb rbac storage openebs helm3 +microk8s enable dns +microk8s enable community +microk8s enable metallb +microk8s enable rbac +microk8s enable storage +microk8s enable openebs +microk8s enable helm3 microk8s status --wait-ready -# + ``` # Add SC4S Helm repo @@ -53,11 +57,7 @@ microk8s helm3 repo update # Create a config file ```yaml -#values.yaml -splunk: - hec_url: "https://10.202.32.101:8088/services/collector/event" - hec_token: "00000000-0000-0000-0000-000000000000" - hec_verify_tls: "yes" +--8<---- "docs/resources/k8s/values_basic.yaml" ``` # Install SC4S @@ -79,12 +79,7 @@ See https://microk8s.io/docs/high-availability Note: Three identically-sized nodes are required for HA ```yaml -#values.yaml -replicaCount: 6 #2x node count -splunk: - hec_url: "https://10.202.32.101:8088/services/collector/event" - hec_token: "00000000-0000-0000-0000-000000000000" - hec_verify_tls: "yes" +--8<---- "docs/resources/k8s/values_ha.yaml" ``` Upgrade sc4s to apply the new config @@ -95,35 +90,8 @@ Using helm based deployment precludes direct configuration of environment variab context files but most configuration can be set via the values.yaml ```yaml -sc4s: - # Certificate as a k8s Secret with tls.key and tls.crt fields - # Ideally produced and managed by cert-manager.io - existingCert: example-com-tls - # - vendor_product: - - name: checkpoint - ports: - tcp: [9000] #Same as SC4S_LISTEN_CHECKPOINT_TCP_PORT=9000 - udp: [9000] - options: - listen: - old_host_rules: "yes" #Same as SC4S_LISTEN_CHECKPOINT_OLD_HOST_RULES=yes - - - name: infoblox - ports: - tcp: [9001, 9002] - tls: [9003] - - name: fortinet - ports: - ietf_udp: - - 9100 - - 9101 - context_files: - splunk_metadata.csv: |- - cisco_meraki,index,foo - host.csv: |- - 192.168.1.1,foo - 192.168.1.2,moon +--8<---- "docs/resources/k8s/values_adv.yaml" + ``` # Resource Management diff --git a/docs/gettingstarted/podman-systemd-general.md b/docs/gettingstarted/podman-systemd-general.md index 7e66fc1cb3..be8241bf9f 100644 --- a/docs/gettingstarted/podman-systemd-general.md +++ b/docs/gettingstarted/podman-systemd-general.md @@ -2,6 +2,8 @@ Refer to [Installation](https://podman.io/getting-started/installation) +NOTE: [READ FIRST (IPv4 forwarding)](./getting-started-runtime-configuration.md#ipv4-forwarding) + # Initial Setup * IMPORTANT: Always use the _latest_ unit file (below) with the current release. By default, the latest container is @@ -10,50 +12,9 @@ that may have been made to the template unit file below (e.g. suggested mount po to relaunching via systemd. * Create the systemd unit file `/lib/systemd/system/sc4s.service` based on the following template: - +#### Unit file ```ini -[Unit] -Description=SC4S Container -Wants=NetworkManager.service network-online.target -After=NetworkManager.service network-online.target - -[Install] -WantedBy=multi-user.target - -[Service] -Environment="SC4S_IMAGE=ghcr.io/splunk/splunk-connect-for-syslog/container2:2" - -# Required mount point for syslog-ng persist data (including disk buffer) -Environment="SC4S_PERSIST_MOUNT=splunk-sc4s-var:/var/lib/syslog-ng" - -# Optional mount point for local overrides and configurations; see notes in docs -Environment="SC4S_LOCAL_MOUNT=/opt/sc4s/local:/etc/syslog-ng/conf.d/local:z" - -# Optional mount point for local disk archive (EWMM output) files -Environment="SC4S_ARCHIVE_MOUNT=/opt/sc4s/archive:/var/lib/syslog-ng/archive:z" - -# Map location of TLS custom TLS -Environment="SC4S_TLS_MOUNT=/opt/sc4s/tls:/etc/syslog-ng/tls:z" - -TimeoutStartSec=0 - -ExecStartPre=/usr/bin/podman pull $SC4S_IMAGE -ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment SC4SHOST=$(hostname -s)" - -ExecStart=/usr/bin/podman run \ - -e "SC4S_CONTAINER_HOST=${SC4SHOST}" \ - -v "$SC4S_PERSIST_MOUNT" \ - -v "$SC4S_LOCAL_MOUNT" \ - -v "$SC4S_ARCHIVE_MOUNT" \ - -v "$SC4S_TLS_MOUNT" \ - --env-file=/opt/sc4s/env_file \ - --health-cmd="/healthcheck.sh" \ - --health-interval=10s --health-retries=6 --health-timeout=6s \ - --network host \ - --name SC4S \ - --rm $SC4S_IMAGE - -Restart=on-abnormal +--8<--- "docs/resources/podman/sc4s.service" ``` * Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication @@ -68,41 +29,12 @@ sudo podman volume create splunk-sc4s-var `/var/lib/containers/storage/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations (typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configuration page for more info. -* Create the subdirectory `/opt/sc4s/local`. This will be used as a mount point for local overrides and configurations. - - * The empty `local` directory created above will populate with defaults and examples at the first invocation -of SC4S for local configurations and context overrides. _Do not_ change the directory structure of -the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout -to read the local configurations properly. See the notes below for which files will be preserved on restarts. - - * In the `local/config/` directory there are four subdirectories that allow you to provide support for device types -that are not provided out of the box in SC4S. To get you started, there is an example log path template (`lp-example.conf.tmpl`) -and a filter (`example.conf`) in the `log_paths` and `filters` subdirectories, respectively. These should _not_ be used directly, -but copied as templates for your own log path development. They _will_ get overwritten at each SC4S start. - - * In the `local/context` directory, if you change the "non-example" version of a file (e.g. `splunk_metadata.csv`) the changes -will be preserved on a restart. - -* Create the subdirectory `/opt/sc4s/archive`. This will be used as a mount point for local storage of syslog events -(if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the "configuration" -document for details on the directory structure the archive uses. - -* Create the subdirectory `/opt/sc4s/tls`. This will be used as a mount point for custom TLS certificates -(if the optional mount is uncommented above). - -* IMPORTANT: When creating the directories above, ensure the directories created match the volume mounts specified in the -unit file above. Failure to do this will cause SC4S to abort at startup. - -# Configure the sc4s environment +* Create subdirectories `/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls` -SC4S is almost entirely controlled through environment variables, which are read from a file at startup. Create a file named -`/opt/sc4s/env_file` and add the following environment variables and values: +Create a file named `/opt/sc4s/env_file` and add the following environment variables and values: ```dotenv -SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://splunk.smg.aws:8088 -SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 -#Uncomment the following line if using untrusted SSL certificates -#SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no +--8<--- "docs/resources/env_file" ``` * Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do _not_ configure HEC @@ -115,58 +47,17 @@ deviate from this. * NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to uncomment the last line in the example above. -## Dedicated (Unique) Listening Ports +For more information about configuration refer to [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations) +and [detailed configuration](../configuration.md). -For certain source technologies, categorization by message content is impossible due to the lack of a unique "fingerprint" in -the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. -For collection of such sources, we provide a means of dedicating a unique listening port to a specific source. - -Follow this step to configure unique ports for one or more sources: - -* Modify the `/opt/sc4s/env_file` file to include the port-specific environment variable(s). Refer to the "Sources" -documentation to identify the specific environment variables that are mapped to each data source vendor/technology. - -## Modify index destinations for Splunk - -Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers. - -* If changes need to be made to index destinations, navigate to the `/opt/sc4s/local/context` directory to start. -* Edit `splunk_metadata.csv` to review or change the index configuration as required for the data sources utilized in your -environment. The key (1st column) in this file uses the syntax `vendor_product`. Simply replace the index value (the 3rd column) in the -desired row with the index appropriate for your Splunk installation. The "Sources" document details the specific `vendor_product` keys (rows) -in this table that pertain to the individual data source filters that are included with SC4S. -* Other Splunk metadata (e.g. source and sourcetype) can be overridden via this file as well. This is an advanced topic, and further -information is covered in the "Log Path overrides" section of the Configuration document. - -## Configure source filtering by source IP or host name - -Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps -apply to support such sources. To identify sources that require this step, refer to the "sources" section of this documentation. See documentation -for your vendor/product to determine if specific configuration is required - -## Configure compliance index/metadata overrides - -In some cases, devices that have been properly sourcetyped need to be further categorized by compliance, geography, or other criterion. -The two files `compliance_meta_by_source.conf` and `compliance_meta_by_source.csv` can be used for this purpose. These operate similarly to -the files above, where the `conf` file specifies a filter to uniquely identify the messages that should be overridden, and the `csv` file -lists one or more metadata items that can be overridden based on the filter name. This is an advanced topic, and further information is -covered in the "Override index or metadata based on host, ip, or subnet" section of the Configuration document. - -## Configure SC4S for systemd and start SC4S +# Configure SC4S for systemd and start SC4S ```bash -sudo systemctl daemon-reload +sudo systemctl daemon-reload sudo systemctl enable sc4s sudo systemctl start sc4s ``` - -# Start SC4S - -```bash -sudo systemctl start sc4s -``` - -# Restart SC4S +## Restart SC4S ```bash sudo systemctl restart sc4s @@ -182,7 +73,7 @@ sudo systemctl enable sc4s sudo systemctl start sc4s ``` -# Stop SC4S +## Stop SC4S ```bash sudo systemctl stop sc4s @@ -204,19 +95,16 @@ This should yield an event similar to the following: syslog-ng starting up; version='3.28.1' ``` -when the startup process proceeds normally (without syntax errors). If you do not see this, +When the startup process proceeds normally (without syntax errors). If you do not see this, follow the steps below before proceeding to deeper-level troubleshooting: * Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443). - * Check to see that the proper indexes are created in Splunk, and that the token has access to them. - * Ensure the proper operation of the load balancer if used. - * Lastly, execute the following command to check the sc4s startup process running in the container. ```bash -podman logs SC4S +docker logs SC4S ``` You should see events similar to those below in the output: @@ -228,7 +116,8 @@ starting goss starting syslog-ng ``` -If you do not see the output above, proceed to the "Troubleshooting" section for more detailed information. +If you do not see the output above, proceed to the ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md) +and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) sections for more detailed information. # SC4S non-root operation @@ -249,10 +138,10 @@ podman system migrate ## Initial Setup -NOTE: Be sure to execute all instructions below as the SC4S user created above with the exception of changes to the unit file, +NOTE: Be sure to execute all instructions below as the SC4S user created above except of changes to the unit file, which requires sudo access. -NOTE2: Using non root prevents the use of standard ports 514 and 601 many device can not alter their destination port this is not +NOTE2: Using non-root prevents the use of standard ports 514 and 601 many device can not alter their destination port this is not a valid configuration for general use, and may only be appropriate for cases where accepting syslog from the public internet can not be avoided. diff --git a/docs/gettingstarted/quickstart_guide.md b/docs/gettingstarted/quickstart_guide.md index 7eef8e5acf..a8c762eba6 100644 --- a/docs/gettingstarted/quickstart_guide.md +++ b/docs/gettingstarted/quickstart_guide.md @@ -41,9 +41,8 @@ netstat -su | grep "receive errors" ``` - * Create the systemd unit file `/lib/systemd/system/sc4s.service`. Copy and paste from the -[SC4S sample unit file](https://splunk-connect-for-syslog.readthedocs.io/en/latest/gettingstarted/podman-systemd-general/#initial-setup -). +* Create the systemd unit file `/lib/systemd/system/sc4s.service`. Copy and paste from the +[SC4S sample unit file (Docker)](docker-systemd-general.md#unit-file) or [SC4S sample unit file (Podman)](podman-systemd-general.md#unit-file) . * Install podman or docker @@ -64,28 +63,26 @@ * Create directories used as a mount point for local overrides and configurations - ``` - mkdir /opt/sc4s/local - mkdir /opt/sc4s/archive - mkdir /opt/sc4s/tls - ``` + ```mkdir /opt/sc4s/local``` + + ```mkdir /opt/sc4s/archive``` + + ```mkdir /opt/sc4s/tls``` * Create the environment file `/opt/sc4s/env_file` and replace the HEC_URL and HEC_TOKEN as appropriate ``` - SC4S_DEST_SPLUNK_HEC_DEFAULT_URL= - SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN= - Uncomment the following line if using untrusted SSL certificates - SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no + --8<--- "docs/resources/env_file" ``` * Configure SC4S for systemd and start SC4S - ``` - sudo systemctl daemon-reload - sudo systemctl enable sc4s - sudo systemctl start sc4s - ``` + ```sudo systemctl daemon-reload ``` + + ```sudo systemctl enable sc4s``` + + ```sudo systemctl start sc4s``` + * Check podman/docker logs for errors (choose one in command below) diff --git a/docs/index.md b/docs/index.md index 8d09239039..71f106aab1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -14,8 +14,7 @@ HTTP event Collector (HEC) rather than writing events to disk for collection by ## Support -* UPDATE! Splunk Connect for Syslog is now officially supported by Splunk. That said, it is still very much an open-source product and -the notes below outlining community support are still highly relevant. +* Splunk Connect for Syslog is an open source project that is now officially supported by Splunk. That said, the notes below outlining community support are still highly relevant. Splunk Connect for Syslog is an open source product developed by Splunkers with contributions from the community of partners and customers. This unique product will be enhanced, maintained and supported by the community, led by Splunkers with deep subject matter expertise. The @@ -26,7 +25,7 @@ Post a question to Splunk Answers using the tag "Splunk Connect For Syslog" Join the #splunk-connect-for-syslog room in the splunk-usergroups Slack Workspace. If you don't yet have an account [sign up](https://docs.splunk.com/Documentation/Community/1.0/community/Chat) -Please use the GitHub issue tracker to submit bugs or request enhancements: https://github.com/splunk/splunk-connect-for-syslog/issues +**Please use the GitHub issue tracker to submit bugs or request enhancements: https://github.com/splunk/splunk-connect-for-syslog/issues** Get involved, try it out, ask questions, contribute filters, and make new friends! diff --git a/docs/performance.md b/docs/performance.md index 9f06108f1d..a3cd904a3f 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -1,28 +1,51 @@ # Performance and Sizing Performance testing against our lab configuration produces the following results and limitations. -## Tested Configuration +## Tested Configurations -* SC4S instance with 2,4,8,12 vCPU using M5zn instances -* Loggen instance m5zn.large -* Single instance Splunk using m5zn.3xlarge +### Splunk Cloud Noah +#### Environment -## Result +* Loggen (syslog-ng 3.25.1) - m5zn.3xlarge +* SC4S(2.30.0) + podman (4.0.2) - m5zn family +* SC4S_DEST_SPLUNK_HEC_WORKERS=10 (default) +* Splunk Cloud Noah 8.2.2203.2 - 3SH + 3IDX +```bash +/opt/syslog-ng/bin/loggen -i --rate=100000 --interval=1800 -P -F --sdata="[test name=\"stress17\"]" -s 800 --active-connections=10 ``` -/opt/syslog-ng/bin/loggen -i --rate=100000 --interval=180 -P -F --sdata="[test name=\"stress17\"]" -s 800 --active-connections=10 hostname 514 - -# m5zn.large 2 8 GiB -average rate = 24077.33 msg/sec, count=4375116, time=181.711, (average) msg size=800, bandwidth=18810.42 kB/sec -# m5zn.xlarge 4 16 GiB -average rate = 38797.44 msg/sec, count=7028962, time=181.171, (average) msg size=800, bandwidth=30310.50 kB/sec -# m5zn.2xlarge 8 32 GiB -average rate = 67252.84 msg/sec, count=12153327, time=180.711, (average) msg size=800, bandwidth=52541.28 kB/sec -# m5zn.3xlarge 12 48 GiB -average rate = 98664.75 msg/sec, count=17834427, time=180.758, (average) msg size=800, bandwidth=77081.84 kB/sec +#### Result + +| SC4S instance | root networking | slirp4netns networking | +|---------------|---------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------| +| m5zn.large | average rate = 21109.66 msg/sec, count=38023708, time=1801.25, (average) msg size=800, bandwidth=16491.92 kB/sec | average rate = 20738.39 msg/sec, count=37344765, time=1800.75, (average) msg size=800, bandwidth=16201.87 kB/sec | +| m5zn.xlarge | average rate = 34820.94 msg/sec, count=62687563, time=1800.28, (average) msg size=800, bandwidth=27203.86 kB/sec | average rate = 35329.28 msg/sec, count=63619825, time=1800.77, (average) msg size=800, bandwidth=27601.00 kB/sec | +| m5zn.2xlarge | average rate = 71929.91 msg/sec, count=129492418, time=1800.26, (average) msg size=800, bandwidth=56195.24 kB/sec | average rate = 70894.84 msg/sec, count=127630166, time=1800.27, (average) msg size=800, bandwidth=55386.60 kB/sec | +| m5zn.2xlarge | average rate = 85419.09 msg/sec, count=153778825, time=1800.29, (average) msg size=800, bandwidth=66733.66 kB/sec | average rate = 84733.71 msg/sec, count=152542466, time=1800.26, (average) msg size=800, bandwidth=66198.21 kB/sec | + + + + +### Splunk Enterprise +#### Environment + +* Loggen (syslog-ng 3.25.1) - m5zn.large +* SC4S(2.30.0) + podman (4.0.2) - m5zn family +* SC4S_DEST_SPLUNK_HEC_WORKERS=10 (default) +* Splunk Enterprise 9.0.0 Standalone + +```bash +/opt/syslog-ng/bin/loggen -i --rate=100000 --interval=600 -P -F --sdata="[test name=\"stress17\"]" -s 800 --active-connections=10 ``` +#### Result + +| SC4S instance | root networking | slirp4netns networking | +|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------| +| m5zn.large | average rate = 21511.69 msg/sec, count=12930565, time=601.095, (average) msg size=800, bandwidth=16806.01 kB/sec
average rate = 21583.13 msg/sec, count=12973491, time=601.094, (average) msg size=800, bandwidth=16861.82 kB/sec | average rate = 20738.39 msg/sec, count=37344765, time=1800.75, (average) msg size=800, bandwidth=16201.87 kB/sec | +| m5zn.xlarge | average rate = 37514.29 msg/sec, count=22530855, time=600.594, (average) msg size=800, bandwidth=29308.04 kB/sec
average rate = 37549.86 msg/sec, count=22552210, time=600.594, (average) msg size=800, bandwidth=29335.83 kB/sec | average rate = 35329.28 msg/sec, count=63619825, time=1800.77, (average) msg size=800, bandwidth=27601.00 kB/sec | +| m5zn.2xlarge | average rate = 98580.10 msg/sec, count=59157495, time=600.096, (average) msg size=800, bandwidth=77015.70 kB/sec
average rate = 99463.10 msg/sec, count=59687310, time=600.095, (average) msg size=800, bandwidth=77705.55 kB/sec | average rate = 84733.71 msg/sec, count=152542466, time=1800.26, (average) msg size=800, bandwidth=66198.21 kB/sec | + -![](throughput.png) ## Guidance on sizing hardware diff --git a/docs/resources/docker-compose.yml b/docs/resources/docker-compose.yml new file mode 100644 index 0000000000..a7400bd80a --- /dev/null +++ b/docs/resources/docker-compose.yml @@ -0,0 +1,29 @@ +version: "3.7" +services: + sc4s: + image: ghcr.io/splunk/splunk-connect-for-syslog/container2:2 + ports: + - target: 514 + published: 514 + protocol: tcp + - target: 514 + published: 514 + protocol: udp + - target: 601 + published: 601 + protocol: tcp + - target: 6514 + published: 6514 + protocol: tcp + env_file: + - /opt/sc4s/env_file + volumes: + - /opt/sc4s/local:/etc/syslog-ng/conf.d/local:z + - splunk-sc4s-var:/var/lib/syslog-ng +# Uncomment the following line if local disk archiving is desired +# - /opt/sc4s/archive:/var/lib/syslog-ng/archive:z +# Map location of TLS custom TLS +# - /opt/sc4s/tls:/etc/syslog-ng/tls:z + +volumes: + splunk-sc4s-var: \ No newline at end of file diff --git a/docs/resources/docker/sc4s.service b/docs/resources/docker/sc4s.service new file mode 100644 index 0000000000..40ae2d661f --- /dev/null +++ b/docs/resources/docker/sc4s.service @@ -0,0 +1,44 @@ +[Unit] +Description=SC4S Container +Wants=NetworkManager.service network-online.target docker.service +After=NetworkManager.service network-online.target docker.service +Requires=docker.service + +[Install] +WantedBy=multi-user.target + +[Service] +Environment="SC4S_IMAGE=ghcr.io/splunk/splunk-connect-for-syslog/container2:2" + +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_MOUNT=splunk-sc4s-var:/var/lib/syslog-ng" + +# Optional mount point for local overrides and configurations; see notes in docs +Environment="SC4S_LOCAL_MOUNT=/opt/sc4s/local:/etc/syslog-ng/conf.d/local:z" + +# Optional mount point for local disk archive (EWMM output) files +Environment="SC4S_ARCHIVE_MOUNT=/opt/sc4s/archive:/var/lib/syslog-ng/archive:z" + +# Map location of TLS custom TLS +Environment="SC4S_TLS_MOUNT=/opt/sc4s/tls:/etc/syslog-ng/tls:z" + +TimeoutStartSec=0 + +ExecStartPre=/usr/bin/docker pull $SC4S_IMAGE + +# Note: /usr/bin/bash will not be valid path for all OS +# when startup fails on running bash check if the path is correct +ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment SC4SHOST=$(hostname -s)" + +ExecStart=/usr/bin/docker run \ + -e "SC4S_CONTAINER_HOST=${SC4SHOST}" \ + -v "$SC4S_PERSIST_MOUNT" \ + -v "$SC4S_LOCAL_MOUNT" \ + -v "$SC4S_ARCHIVE_MOUNT" \ + -v "$SC4S_TLS_MOUNT" \ + --env-file=/opt/sc4s/env_file \ + --network host \ + --name SC4S \ + --rm $SC4S_IMAGE + +Restart=on-abnormal \ No newline at end of file diff --git a/docs/resources/env_file b/docs/resources/env_file new file mode 100644 index 0000000000..d3d837fdf7 --- /dev/null +++ b/docs/resources/env_file @@ -0,0 +1,4 @@ +SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://your.splunk.instance:8088 +SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx +#Uncomment the following line if using untrusted SSL certificates +#SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no \ No newline at end of file diff --git a/docs/resources/k8s/values_adv.yaml b/docs/resources/k8s/values_adv.yaml new file mode 100644 index 0000000000..cb6380abcf --- /dev/null +++ b/docs/resources/k8s/values_adv.yaml @@ -0,0 +1,29 @@ +sc4s: + # Certificate as a k8s Secret with tls.key and tls.crt fields + # Ideally produced and managed by cert-manager.io + existingCert: example-com-tls + # + vendor_product: + - name: checkpoint + ports: + tcp: [9000] #Same as SC4S_LISTEN_CHECKPOINT_TCP_PORT=9000 + udp: [9000] + options: + listen: + old_host_rules: "yes" #Same as SC4S_LISTEN_CHECKPOINT_OLD_HOST_RULES=yes + + - name: infoblox + ports: + tcp: [9001, 9002] + tls: [9003] + - name: fortinet + ports: + ietf_udp: + - 9100 + - 9101 + context_files: + splunk_metadata.csv: |- + cisco_meraki,index,foo + host.csv: |- + 192.168.1.1,foo + 192.168.1.2,moon \ No newline at end of file diff --git a/docs/resources/k8s/values_basic.yaml b/docs/resources/k8s/values_basic.yaml new file mode 100644 index 0000000000..495a513413 --- /dev/null +++ b/docs/resources/k8s/values_basic.yaml @@ -0,0 +1,5 @@ +#values.yaml +splunk: + hec_url: "https://xxx.xxx.xxx.xxx:8088/services/collector/event" + hec_token: "00000000-0000-0000-0000-000000000000" + hec_verify_tls: "yes" \ No newline at end of file diff --git a/docs/resources/k8s/values_ha.yaml b/docs/resources/k8s/values_ha.yaml new file mode 100644 index 0000000000..b9d1f3ca32 --- /dev/null +++ b/docs/resources/k8s/values_ha.yaml @@ -0,0 +1,6 @@ +#values.yaml +replicaCount: 6 #2x node count +splunk: + hec_url: "https://xxx.xxx.xxx.xxx:8088/services/collector/event" + hec_token: "00000000-0000-0000-0000-000000000000" + hec_verify_tls: "yes" \ No newline at end of file diff --git a/docs/resources/podman/sc4s.service b/docs/resources/podman/sc4s.service new file mode 100644 index 0000000000..8c2c7e1f98 --- /dev/null +++ b/docs/resources/podman/sc4s.service @@ -0,0 +1,45 @@ +[Unit] +Description=SC4S Container +Wants=NetworkManager.service network-online.target +After=NetworkManager.service network-online.target + +[Install] +WantedBy=multi-user.target + +[Service] +Environment="SC4S_IMAGE=ghcr.io/splunk/splunk-connect-for-syslog/container2:2" + +# Required mount point for syslog-ng persist data (including disk buffer) +Environment="SC4S_PERSIST_MOUNT=splunk-sc4s-var:/var/lib/syslog-ng" + +# Optional mount point for local overrides and configurations; see notes in docs +Environment="SC4S_LOCAL_MOUNT=/opt/sc4s/local:/etc/syslog-ng/conf.d/local:z" + +# Optional mount point for local disk archive (EWMM output) files +Environment="SC4S_ARCHIVE_MOUNT=/opt/sc4s/archive:/var/lib/syslog-ng/archive:z" + +# Map location of TLS custom TLS +Environment="SC4S_TLS_MOUNT=/opt/sc4s/tls:/etc/syslog-ng/tls:z" + +TimeoutStartSec=0 + +ExecStartPre=/usr/bin/podman pull $SC4S_IMAGE + +# Note: /usr/bin/bash will not be valid path for all OS +# when startup fails on running bash check if the path is correct +ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment SC4SHOST=$(hostname -s)" + +ExecStart=/usr/bin/podman run \ + -e "SC4S_CONTAINER_HOST=${SC4SHOST}" \ + -v "$SC4S_PERSIST_MOUNT" \ + -v "$SC4S_LOCAL_MOUNT" \ + -v "$SC4S_ARCHIVE_MOUNT" \ + -v "$SC4S_TLS_MOUNT" \ + --env-file=/opt/sc4s/env_file \ + --health-cmd="/healthcheck.sh" \ + --health-interval=10s --health-retries=6 --health-timeout=6s \ + --network host \ + --name SC4S \ + --rm $SC4S_IMAGE + +Restart=on-abnormal \ No newline at end of file diff --git a/docs/sources/index.md b/docs/sources/index.md index 5ed74a0941..1d7383c37e 100644 --- a/docs/sources/index.md +++ b/docs/sources/index.md @@ -144,7 +144,8 @@ block parser cisco_ios_debug-postfilter() { }; application cisco_ios_debug-postfilter[sc4s-postfilter] { filter { - "${fields.sc4s_vendor_product}" eq "cisco_ios" + "${fields.sc4s_vendor}" eq "cisco" and + "${fields.sc4s_product}" eq "ios" #Note regex reads as # start from first position # Any atleast 1 char that is not a `-` diff --git a/mkdocs.yml b/mkdocs.yml index 00bf381e01..082497d8c8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -13,6 +13,8 @@ markdown_extensions: - fenced_code - sane_lists - codehilite + - pymdownx.snippets + theme: name: "material" @@ -24,28 +26,32 @@ theme: nav: - Home: "index.md" - - "Architectural Considerations": "architecture.md" + - Architectural Considerations: "architecture.md" - Load Balancers: "lb.md" - Getting Started: - - "Read First": "gettingstarted/index.md" - - "Podman + systemd": "gettingstarted/podman-systemd-general.md" - - "Docker CE + systemd": "gettingstarted/docker-systemd-general.md" - - "MicroK8s + Linux": "gettingstarted/k8s-microk8s.md" - - "Docker Compose": "gettingstarted/docker-swarm-general.md" - - "Docker Desktop + Compose (MacOS)": "gettingstarted/docker-compose-MacOS.md" - - "Bring your own Envionment": "gettingstarted/byoe-rhel8.md" - - "Quickstart Guide": "gettingstarted/quickstart_guide.md" + - Read First: "gettingstarted/index.md" + - Splunk Setup: "gettingstarted/getting-started-splunk-setup.md" + - Runtime Configuration: "gettingstarted/getting-started-runtime-configuration.md" + - Quickstart Guide: "gettingstarted/quickstart_guide.md" + - Select Runtime: + - Podman + systemd: "gettingstarted/podman-systemd-general.md" + - Docker CE + systemd: "gettingstarted/docker-systemd-general.md" + - MicroK8s + Linux: "gettingstarted/k8s-microk8s.md" + - Docker Compose: "gettingstarted/docker-compose.md" + - Docker Desktop + Compose (MacOS): "gettingstarted/docker-compose-MacOS.md" + - Bring your own Envionment: "gettingstarted/byoe-rhel8.md" + - Docker&Podman offline installation: "gettingstarted/docker-podman-offline.md" - Configuration: "configuration.md" - Development: "developing/index.md" - Destinations: "destinations.md" - Sources: - - Read First: sources/index.md - - Basic Onboarding: sources/base - - Known Vendors: sources/vendor + - Read First: "sources/index.md" + - Basic Onboarding: "sources/base" + - Known Vendors: "sources/vendor" - Performance: "performance.md" - Troubleshooting: - SC4S Startup and Validation: "troubleshooting/troubleshoot_SC4S_server.md" - SC4S Logging and Troubleshooting Resources: "troubleshooting/troubleshoot_resources.md" - Experiments: "experiments.md" - - "Upgrading SC4S": "upgrade.md" - - "SC4S FAQ": "faq.md" + - Upgrading SC4S: "upgrade.md" + - SC4S FAQ: "faq.md"