Spike - Wazuh kubernetes

[teddytpc1]

Objective
https://github.com/wazuh/internal-devel-requests/issues/1319

Description

As part of the DevOps overhaul objective we need to conduct research, analyze alternatives, and design how to implement the following changes.

1. Repository Scope Clarification:

• The Wazuh Kubernetes repository should focus solely on deployments.
• Analyze improvements for the Wazuh Kubernetes deployment, including an analysis of the Ingress controller and Load Balancer optimization.
• Leverage the Docker images improvements to use health checks for the different pods.

2. Deployment Simplification:

• Utilize out-of-the-box Wazuh configurations for deployments.

3. Testing Improvements:

• Enhance deployment tests with additional checks, including log validation for errors and warnings.
• The test workflows must work with development images.
• The test workflows must allow using specific image tags for each component for manual workflow executions.

4. Documentation Updates:

• Simplify/improve the Kubernetes installation documentation.

Implementation restrictions

• Testing Environment: The tests must be implemented using GitHub Actions (GHA).
• Compatibility: The workflow should be compatible with the environments used for PR testing and manual testing.
• Logs Validation: The logs checking must identify and report critical issues (e.g., errors, warnings) in a clear and actionable way.
• Minimal Maintenance: The implementation should aim for low complexity and minimal maintenance overhead.

Plan

TBD

[vcerenu]

I have been investigating the service options to use to deploy Wazuh, eliminating the use of AWS NLBs and thereby simplifying the use of cloud resources.

[vcerenu]

Conclusions

1. Repository Scope Clarification:

• The Wazuh Kubernetes repository should focus solely on deployments.
• Analyze improvements for the Wazuh Kubernetes deployment, including an analysis of the Ingress controller and Load Balancer optimization.
• Leverage the Docker images improvements to use health checks for the different pods.

Currently the Kubernetes repository is focused solely on deployments, so we do not have to split it up.
The Load Balancer service in Kubernetes is vendor agnostic, so its use depends on the need we have when deploying the components.
The option to use Load Balancer is a design issue. The need for load balancing can be solved with a Load Balancer or by adding an Ingress Controller that takes care of this work, both solutions work and will depend on the client's infrastructure. As an example of deployment, I see the use of Load Balancers as convenient, because the purpose of the services is better understood and requires fewer dependencies at the time of a test.
It is also necessary to review what types of connections Wazuh Manager will require in the version to be able to determine what type of Load Balancer is the most convenient for the connections, since currently in AWS we are using a CLB, but we should migrate this deployment to use an NLB or an ALB.
Regarding the health checks of the pods, we can add liveness and readiness probes, which perform small checks before labeling a pod as healthy and also the need to restart it. These health checks can also be useful for the tests we perform, since we can use all the deployment review logic in each deployed pod and thus be able to trust the pod's status better, without having to depend on external controls.

2. Deployment Simplification:

• Utilize out-of-the-box Wazuh configurations for deployments.

Regarding the simplification of the implementation, we have to take into account that Kubernetes is responsible for deploying all the components that normally correspond to both software and hardware in a deployment in VMs, so its simplification depends more on the purpose of the deployment itself. If we want to have a product ready to run, we have to add all the necessary components for it, such as deployments, statefulset, services, secrets and others, which must have the default parameters in many cases and depending on the Docker images we have, the possibility of adding parameters that allow the configuration of the deployment. Currently we have a basic scheme that differentiates a deployment in a cluster installed by itself and a deployment that customizes many parameters for AWS services. We need to check if we are going to keep these customizations or if we are only going to provide a base that can be adapted by the user to the environment he wants and if we are going to request dependencies for the deployment, such as the use of network load balancers in AWS or the use of Ingress Controller.

3. Testing Improvements:

• Enhance deployment tests with additional checks, including log validation for errors and warnings.
• The test workflows must work with development images.
• The test workflows must allow using specific image tags for each component for manual workflow executions.

We currently have a test workflow, which has several checks but requires adding some more, such as error controls in logs. This workflow only uses images published on Docker Hub, so it requires having images in those image repositories. We must adapt these tests to use images from private repositories, which is possible and the use of these repositories was successfully analyzed. Regarding the issue of using specific image tags, it is required to add the logic within the test.

4. Documentation Updates:

• Simplify/improve the Kubernetes installation documentation.

Regarding the Kubernets documentation, it currently contains the necessary parameters for the deployment itself, but it is missing information regarding the possibility of adding integrations and other configurations that require additional software, such as AWS CLI for integrations with AWS, an SMTP client for sending emails, etc. We should plan what type of information we require in our documentation, not duplicate data that may already be in the documentation for each component and, if necessary, add links to the software and dependencies that we require. ex: Kubectl and Kustomize.
We should also provide more information regarding the communication required by the Wazuh components, in order to help users build their own deployments based on the base we provide.

Currently we do not have all the information regarding the communication protocols that Wazuh will require for the connection, such as the internal processes that would allow us to determine the tests with greater precision, so this issue is blocked until we receive this information.

[teddytpc1]

Update

We need to develop a plan with all the items from the analysis. The plan must be ordered and each task must have an Owner and the teams involved.
After the plan is validated, we should create the corresponding issues to start working on each task.