Skip to content

Home

Jump to bottom
4Quarks edited this page Sep 18, 2024 · 36 revisions

pDNSSOC is a minimalistic component allowing DNS data to be centrally collected, and correlated with malicious domains / IPs from a MISP instance.

Installation overview

A complete pDNSSOC deployment requires:

  • Access to a MISP instance (URL + API key are needed)
  • A source of DNS data, for example:
    • dnstap files (typically rsync'ed of SSH)
    • a DNS server with a go-dnscollector sender
  • A pDNSSOC server: a go-dnscollector receiver + pDNSSOC

pDNSSOC server

go-dnscollector receiver

  1. Create the user that will run the service:

    sudo useradd --system --no-create-home --shell /usr/sbin/nologin dnscollector

  2. Define the version of the go-dnscollector that you want to install:

     GO_DNSCOLLECTOR_VERSION=$(curl -s https://api.github.com/repos/dmachard/go-dnscollector/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/' | sed 's/^v//')

  3. Install go-dnscollector

    curl -LO  "https://github.com/dmachard/go-dnscollector/releases/download/v${GO_DNSCOLLECTOR_VERSION}/go-dnscollector_${GO_DNSCOLLECTOR_VERSION}_linux_amd64.tar.gz" && \
tar xvf "go-dnscollector_${GO_DNSCOLLECTOR_VERSION}_linux_amd64.tar.gz" && \
mv go-dnscollector /usr/bin/

  4. Adjust the permissions for the user and create the directories needed to be able to run go-dnscollector:

    chmod +x /usr/bin/go-dnscollector
chcon -t bin_t /usr/bin/go-dnscollector
setcap cap_net_raw+ep /usr/bin/go-dnscollector
chown dnscollector:dnscollector /usr/bin/go-dnscollector

mkdir -p /var/dnscollector /var/dnscollector/alerts /var/dnscollector/queries /var/dnscollector/matches
touch /var/dnscollector/alerts/matches.json /var/dnscollector/misp_ips.txt /var/dnscollector/misp_domains.txt /var/dnscollector/queries/queries.json /var/dnscollector/alerts_db.txt /var/dnscollector/matches/matches_domains.json /var/dnscollector/matches/matches_ips.json
chown -R dnscollector:dnscollector /var/dnscollector/
chmod -R u+rw /var/dnscollector/

  5. Adjust the configuration file, which is automatically generated as config.yml using the following templates:

    mkdir -p /etc/dnscollector 
curl -o /etc/dnscollector/config.yml https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/dnscollector/server.yml
chown -R dnscollector:dnscollector /etc/dnscollector/
chmod -R u+rw /etc/dnscollector/ 
vi /etc/dnscollector/config.yml

  6. Test the configuration file to make sure it doesn't have typos:

    go-dnscollector -config /etc/dnscollector/config.yml -test-config

  7. Execute the collector:

Configure the collector as a service:

 curl -o /etc/systemd/system/dnscollector.service https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/dnscollector/dnscollector.service
 systemctl daemon-reload
 systemctl start dnscollector
 systemctl enable dnscollector

For debugging purposes, it is possible to start go-dnscollector manually

 go-dnscollector -config /etc/dnscollector/config.yml
  1. Ensure that the collecting port set in the configuration file is accessible and the port open. For example: 
    sudo firewall-cmd --zone=public --add-port=7001/tcp --permanent
sudo firewall-cmd --zone=public --add-port=7001/udp --permanent
sudo firewall-cmd --reload

pDNSSOC

  1. Create the pdnssoc directory.

    mkdir /etc/pdnssoccli

  2. Install pdnssoc-cli

    pip3 install git+https://github.com/safer-trust/pdnssoc-cli.git@topic/packaging

  3. Create the pdnssoccli configuration file (pdnssoccli.yml) under /etc/pdnssoccli and modify it based on the pdnssoccli template. For details, see the configuration documentation.

    curl -o /etc/pdnssoccli/pdnssoccli.yml https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/pdnssoccli/pdnssoccli.yml
chown -R dnscollector:dnscollector /etc/pdnssoccli
vi /etc/pdnssoccli/pdnssoccli.yml
  1. Create the cronjob to automate correlation and alerting: 
    curl -o /etc/cron.d/pdnssoccli https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/pdnssoccli/pdnssoccli.cron
service crond reload

go-dnscollector sender (DNS server side)

  1. Create the user that will run the service:

    sudo useradd --system --no-create-home --shell /usr/sbin/nologin dnscollector

  2. Define the version of the go-dnscollector that you want to install:

     GO_DNSCOLLECTOR_VERSION=$(curl -s https://api.github.com/repos/dmachard/go-dnscollector/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/' | sed 's/^v//')

  3. Install go-dnscollector

    curl -LO  "https://github.com/dmachard/go-dnscollector/releases/download/v${GO_DNSCOLLECTOR_VERSION}/go-dnscollector_${GO_DNSCOLLECTOR_VERSION}_linux_amd64.tar.gz" && \
tar xvf "go-dnscollector_${GO_DNSCOLLECTOR_VERSION}_linux_amd64.tar.gz" && \
mv go-dnscollector /usr/bin/

  4. Adjust the permissions for the user and create the directories needed to be able to run go-dnscollector:

    chmod +x /usr/bin/go-dnscollector
chcon -t bin_t /usr/bin/go-dnscollector
setcap cap_net_raw+ep /usr/bin/go-dnscollector
chown dnscollector:dnscollector /usr/bin/go-dnscollector

mkdir -p /var/dnscollector
chown -R dnscollector:dnscollector /var/dnscollector/
chmod -R u+rw /var/dnscollector/

  5. Adjust the configuration file, which is automatically generated as config.yml using the following templates:

    mkdir -p /etc/dnscollector 
curl -o /etc/dnscollector/config.yml https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/dnscollector/client.yml
chown -R dnscollector:dnscollector /etc/dnscollector/
chmod -R u+rw /etc/dnscollector/ 
vi /etc/dnscollector/config.yml

  6. Test the configuration file to make sure it doesn't have typos:

    go-dnscollector -config /etc/dnscollector/config.yml -test-config

  7. Execute the collector:

Configure the collector as a service:

 curl -o /etc/systemd/system/dnscollector.service https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/dnscollector/dnscollector.service
 systemctl daemon-reload
 systemctl start dnscollector
 systemctl enable dnscollector

For debugging purposes, it is possible to start go-dnscollector manually

 go-dnscollector -config /etc/dnscollector/config.yml

Testing and debugging pDNSSOC

  • Check if the service is running and its logs:
# systemctl status dnscollector
# journalctl -u dnscollector -f
  • Check if the process is running:
# ps -aux | grep dnscollector
dnscoll+   37571  0.0  1.1 1395372 40108 ?       Ssl  May23   2:03 /usr/bin/go-dnscollector -c /etc/dnscollector/config.yml

  • Check if the connection has been established. We will use the port 7001 as example:

    A. From the DNS sensor:

    # netstat -putan | grep 7001
tcp        0      0 IP_DNS:59450            IP_PDNSSOC:7001         ESTABLISHED 37571/go-dnscollect

    B. From pDNSSOC:

    # netstat -putan | grep 7001
tcp6       0      0 :::7001                 :::*                    LISTEN      19378/go-dnscollect 
tcp6       0      0 IP_PDNSSOC:7001         IP_DNS:59450            ESTABLISHED 19378/go-dnscollect

  • Check if the pDNSSOC collector is receiving logs:

# tail /var/dnscollector/queries.json
# tcpdump -i eth0 -A port 7001
  • Check the location where pDNSSOC was installed:
# pip show pdnssoc-cli
  • Verify that the pDNSSOC command line tool works
# pdnssoc-cli fetch-iocs
# pdnssoc-cli correlate
# pdnssoc-cli alert
  • Verify that pdnssoccli.yml has a right format
# pip install yamllint
# yamllint /etc/pdnssoccli/pdnssoccli.yml
  • Check if the IOCs could be fetched from MISP
# tail /var/dnscollector/misp_domains.txt 
# tail /var/dnscollector/misp_ips.txt
  • Test a malicious domain resolution to check that the alerts are created. You can use any domain from the misp_domains.txt,
# dig @IP_DNS maliciousdomain.com
# cat /var/dnscollector/queries.json | grep maliciousdomain.com

Getting alerts

email

  • Need to download the template from github and put it in /etc/pdnssoccli as a file called alert_email.html

slack

  • Add your slack webhook to the /etc/pdnssoccli/pdnssoccli.yml file (template can be found here)

mattermost

  • First create a webhook for mattermost:
    • Start up mattermost and go to the team where you want to recieve the alerts
    • In the top left corner you will see an icon which looks like a square grid (above the team icons), click on this
    • A dropdown should appear, from here click 'Intergrations'
    • Select 'Incoming Webhooks'
    • 'Add Incoming Webhook'
    • Fill out the form and save the webhook, you should then receive a url, copy this to your clipboard
    • Now go to your pDNSSOC machine and edit the pdnssoc config file: vi /etc/pdnssoccli/pdnssoccli.yml
    • Under the 'alerting' header, add the following configuration: 
      slack:
  slack_hook: <your mattermost webhook url>
    • Now you can use the command pdnssoc-cli alert and any new alerts should be sent automatically to the mattermost channel you configured

telegram

  • Create a telegram bot which will be used for the pDNSSOC alerts:
    • Search for 'BotFather' in telegram and write '/newbot' in the chat
    • Follow the instructions and keep note of the API key given to you once the bot is created
    • Add the bot to whichever telegram channel you would like to receive alerts
    • Once the bot has been added, find the telegram channel ID. This can be done by accessing the telegram channel in the browser and copying the end digits (they should be in this format, including the hyphen: -0000000000). For a more in-depth tutorial on finding the telegram channel ID, click here. Once you have found the telegram channel ID, you must add the string '100' between the hyphen and the rest of the ID, like this: Original = -9999999999 After = -1009999999999
  • Now that you have your telegram bot and API key, you need to change the configuration file of your pdnssoc-cli program:
    • vi /etc/pdnssoccli/pdnssoccli.yml
    • Add the following under the 'alerting' tab of the pdnssoc-cli config file (remember the indentation is important): 
        telegram:
    telegram_chat_id: <your_telegram_chat_ID>
    telegram_bot_token: <your_telegram_bot_API_token>
  • You can test this has worked by trying to send an alert using the command pdnssoc-cli alert

Updating the version

  • If you have already installed pdnssoc-cli on your machine you will need to run the commands below to update it:

    git clone https://github.com/CERN-CERT/pdnssoc-cli.git
cd pdnssoc-cli/
python3.11 -m pip install --force-reinstall .

DNS Server (for testing)

To set up a test environment, you can easily deploy a Bind9 DNS server by following the steps outlined below. Please note that the provided template and installation instructions are intended for testing purposes only and are NOT suitable for a production environment. For best practices and production setups, please refer to the Official Documentation.

  1. Install Bind9 
    dnf copr enable isc/bind
yum install isc-bind
  2. Create log directory and edit the name.conf file using the template 
    mkdir -p /var/log/named 
chown named:named /var/log/named 
curl -o /etc/opt/isc/scls/isc-bind/named.conf https://raw.githubusercontent.com/CERN-CERT/pDNSSOC/main/files/configuration/test_lab/named.conf
vim /etc/opt/isc/scls/isc-bind/named.conf
chown named:named /etc/opt/isc/scls/isc-bind/named.conf
sudo -u named /opt/isc/isc-bind/root/usr/bin/named-checkconf /etc/opt/isc/scls/isc-bind/named.conf
  3. Start and enable the DNS service 
    systemctl start isc-bind-named
systemctl enable isc-bind-named
systemctl status isc-bind-named
  4. Open the internal firewall if you want to resolve domains from other instances 
    systemctl start firewalld
systemctl enable firewalld
firewall-cmd --permanent --add-service=dns
firewall-cmd --reload
firewall-cmd --list-all

Test installation

  • Check that the DNS can resolve domains.
host1# dig @IP_DNS maliciousdomain.com
dns# /opt/isc/isc-bind/root/usr/bin/dnstap-read /var/log/named/dnstap.log 
27-May-2024 17:57:01.255 CQ IP_HOST1:47263 -> IP_DNS:53 UDP 49b maliciousdomain.com/IN/A
Clone this wiki locally