Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support conditions based autodiscover for Docker containers for Fleet managed Agents #5532

Open
pgeyman opened this issue Sep 12, 2024 · 1 comment
Labels
Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Comments

@pgeyman
Copy link

pgeyman commented Sep 12, 2024

Describe the enhancement:
There is documentation that describes how to configure standalone Elastic Agents with autodiscover, but nothing about how to use this for Fleet Managed agents.

Provide support for configuring conditions based autodiscover for Docker containers for Fleet managed Agents that will allow processing the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App).

If this functionality does exist, please provide documentation on how to configure it for Fleet Managed Agents.

Describe a specific use case for the enhancement or feature:
We are currently using Filebeat with autodiscover to send logs from Docker containers of different flavours (Apache HTTP, MySQL, etc).

We are now trying to migrate over to Elastic Agent with Fleet managed agents, but I cannot find any documentation or examples of how to configure the agents with an equivalent setup to what we have with Filebeat and autodiscover.

We need to be able to setup a Fleet Agent Policy that will process the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App)

Example filebeat autodiscover setup:

filebeat.autodiscover:
  # List of enabled autodiscover providers
  providers:
    - type: docker
      templates:
        - condition:
            equals.docker.container.labels.com.docker.compose.service: apache
          config:
            - module: apache
              access:
                input:
                  type: container
                  stream: stdout
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              error:
                input:
                  type: container
                  stream: stderr
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
        - condition:
            equals.docker.container.labels.com.docker.compose.service: webapp
          config:
            - type: container
              paths:
                - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
              multiline.negate: false
              multiline.match: after
        - condition:
            equals.docker.container.labels.com.docker.compose.service: mysql
          config:
            - module: mysql
              error:
                input:
                  type: container
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              slowlog: 
                enabled: false

What is the definition of done?
Being able to setup a Fleet Agent Policy that will process the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App)?

@cmacknz cmacknz added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Sep 12, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

No branches or pull requests

3 participants