Ship Scan Results to ElasticSearch

[dfoley84]

Hi All,

I'm unable to find in the Doc how to configure the scan results to ship to ElasticSearch.
can someone point me to the right direction on how to get this configured.

Thanks;

[toniblyx]

Hello @dfoley84, using S3 bucket Event notifications and the Prowler option to save results in a bucket it is possible.
prowler <provider> -M json -B my-bucket/folder/ more info here https://docs.prowler.cloud/en/latest/tutorials/reporting/#send-report-to-aws-s3-bucket

Here's an example of a lambda function in Python that retrieves a JSON file from an S3 bucket and ingests it into an Elasticsearch cluster using the boto3 library:

import json
import boto3
from elasticsearch import Elasticsearch

s3 = boto3.resource('s3')
es = Elasticsearch(hosts=[{'host': '<your-elasticsearch-host>', 'port': 443}], use_ssl=True)

def lambda_handler(event, context):
    # Get the object from the event and show its content type
    bucket_name = event['Records'][0]['s3']['bucket']['name']
    object_key = event['Records'][0]['s3']['object']['key']
    obj = s3.Object(bucket_name, object_key)
    file_content = obj.get()['Body'].read().decode('utf-8')

    # Parse the JSON file
    json_data = json.loads(file_content)

    # Ingest the JSON data into Elasticsearch
    for item in json_data:
        es.index(index='<your-index-name>', doc_type='<your-doc-type>', body=item)

    return {
        'statusCode': 200,
        'body': 'Successfully ingested data into Elasticsearch'
    }

Note that you will need to replace <your-elasticsearch-host>, <your-index-name>, and <your-doc-type> with the appropriate values for your Elasticsearch cluster.

Something like this should work, we don't have that section in docs yet, if you confirm it works we will add it. Thanks.

[rubtoa]

You might also consider standard log shippers (fluentd, logstash for example). It's already JSON so not a lot work in terms of formatting