modsecurityhelpers

Helpers for modsecurity Logs

The helpers at the moment:

• 1 additional rule set that logs the anomaly score even when the HTTP request isn't intercepted
• 1 interactive script that greps and filters logs for relevant data (feel free to set $logpath to whatever time frame is needed as it greps recursively from $logpath)

The story

• when modsecurity is set to "on" but with a high anomaly threshold in order to NOT intercept the request (kind of like audit mode) it does not actually write the anomaly score into the log file.

  The rules in ruleset_add_anomaly_score.after.conf changes this.

• when in high-threshold-audit-mode: this ruleset is needed for the script moi.sh to work, as it greps for the added log content.

• it can be quite tricky to obtain relevant information from the modsecurity logs, the interactive script can help here

Requirements

• modsecurity v2

• CoreRuleSet

  tested with SecRuleEngine set to on with rules 2001099 and 2001100 and high threshold level

  tested with SecRuleEngine set to on with intercepted requests (works without ruleset 2001099/2001100)

• if needed for high-threshold-audit-mode: Rules 2001099/2001100 that run AFTER the rules of CoreRuleSet (defined in apache-mods/security2.conf)

• SecAuditLogType should be set to "Concurrent" in /etc/modsecurity/modsecurity.conf

     SecAuditLogType Concurrent
     SecAuditLogStorageDir /var/log/modsec_audit

• dialog (sudo apt install dialog)

Tested with

• modsecurity 2.9.2
• CoreRuleSet 3.3.2
• Ubuntu 18.04/20.04
• Apache

INSTRUCTION

Clone Repository and add executable bit:

git clone https://github.com/stefanpinter/modsecurityhelpers.git
cd modsecurityhelpers
chmod +x moi.sh

Run this script

 ./moi.sh

It searches for the messages in the logs. The messages and their number of occurence is shown. Choose the message you want info about.

It will present you

• the matching logfiles
• information about the request header (for example GET /drupal/setup.php)
• the PTR record of the IP of the http client (if there is one)
• the id of the matchin rule
• information about what matched

Filter when using Executing Paranoia Level

If you use EXECUTING Paranoia level, this might be handy:

-p2	Filter for paranoia level 2 Messages
-p3	Filter for paranoia level 3 Messages
-p4	Filter for paranoia level 4 Messages

This is useful in the following case:

Paranoia Level is set to 1

Executing Paranoia Level is set to 2

Now a http request with the following scores:

Paranoia level 1 score = 0

Paranoia level 2 score = 5 (remember PL2 is only set to "executing")

Paranoia level 3 score = 0

Paranoia level 4 score = 0

Use -p2 to show paranoia level 2 requests. Not using -p2 will show nothing, because paranoia level 1 score is 0 and moi by default only shows logs of requests that have been blocked (or would have been blocked but weren't because of high threshold mode). -pl3 and -pl4 will show nothing.

Remember to clear cache with "moi -r"

Change Time Frame

Use -l (smaller case L)

Examples:

./moi.sh -l "/var/log/modsec_audit/www-data/"

or

./moi.sh -l .

(potentially) FAQ

• moi.sh quits unexpectedly

if it quits unexpectedly it probably did not find logs of the chosen host where it (would have) intercepted

Maybe try clearing the TSCS cache (tremendously sophisticated caching system)

• moi.sh does not show the hosts that I have expected. Why?

you should try to clear the TSCS cache (tremendously sophisticated caching system) by launching moi.sh like this

./moi.sh -r

• There are soo many logs generated by our website scanner that I want moi.sh to ignore. How can moi.sh help here?

Let's say you use Burpsuite to scan your website and it sends a "User-Agent"-header that contains "fakechromiumengionebyburpsuite". Just run moi.sh like so:

./moi.sh -i "fakechromiumengionebyburpsuite"

You can also use IP-Adresses or use several strings that are combined with regex ORs -> |

./moi.sh -i "fakechromiumengineobyburpsuite|80.1.45.99"

• I always forget those easy arguments and options. How can moi.sh help me?

moi.sh comes with an integrated help document that can be called by doing something wrong or by calling it via -h

./moi.sh -h

TODO

• add features reagarding paranoia level (show paranoia level somewhere)
• show anomaly score of each request
• add the possibility to write the result to a textfile
• bugfixes
• further caching
• work with compressed files