Fail2stat

The Idea

I will explain how to combine your favorite website statistics software with fail2ban in order to produce statistics for break-in attempts at your server. The idea is simple: let fail2ban convert any ban triggering line in a log file to a web server log file format; let the newly created pseudo webserver log file get examined by your website statistics software. Here I treat the special case of an Apache web servers running with Webalizer.

The converter

Adapt the following script to your needs and save it, executable by the fail2ban user, in /usr/local/bin/fail2stat.

#!/bin/bash
# fail2stat
#       makes fail2ban events processable for webalizer
#
IP="$1"
NAME="$2"
LOGPATH="$3"
YEAR="$(date +%Y)"
# get the log lines
grep $IP $LOGPATH > /tmp/fail2stat
# don't process if they are from an apache log file
if [[ "$LOGPATH" == *"apache"* ]]; then cat /tmp/fail2stat >> /var/log/fail2stat.log; rm /tmp/fail2stat; exit 0; fi
# processing in the following way is valid for auth.log and mail.log; 
# adapt if you need differently formatted logs
while read MONTH DAY TIME TARGET DAEMON ETC;
        do
                # as the year is not part of the log line, we have to take extra care around New Year.
                MONTH_NUMBER="$(date -d "$MONTH 1, $YEAR" +%m)"
                if [[ $((MONTH_NUMBER-1)) -ge $(date +%m) ]]; then YEAR=$((YEAR -1)); fi
                #
                DAEMON="${DAEMON%%[*}"
                MSG="${ETC#*: }"; MSG="${MSG%%from *}"
                MSG="$(echo $MSG | sed 's/ /_/g')"
                USER="${MSG##*user?}"; USER="${USER%%_*}";
                # the numbers 200 and 1024 are arbitrary
                echo "$IP - $USER [$DAY/$MONTH/$YEAR:$TIME +0000] \"GET /$NAME/$MSG/ HTTP/1.1\" 200 1024 \"-\" \"$TARGET/$DAEMON\"" >> /var/log/fail2stat.log
        done < /tmp/fail2stat
rm /tmp/fail2stat
exit 0

fail2ban configuration

jail.local

Append the following lines to /etc/fail2ban/jail.local under [Default] > ACTIONS. First, define the statistics action

# statistics action
stat = webalizer

Append the statistics action at your favorite action shortcut, e.g. create

# ban & send & log stats
action_mwls = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
               %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath="%(logpath)s", chain="%(chain)s"]
               %(stat)s[name=%(__name__)s,logpath="%(logpath)s"]

and, finally, set the default action to include statistics:

action = %(action_mwls)s

If you have defined jails with individual actions, take care to include the statistics action there as well.

action.d/webalizer.conf

Copy a standard configuration file and change the line defining actionban to

actionban = /usr/local/bin/fail2stat <ip> <name> <logpath>

The complete file might look like this (to be uploaded).

Now it is time to restart fail2ban.

apache configuration

Set up a new virtual host. The configuration in /etc/apache2/sites-available/fail2stat.conf might be like this:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName yourserver.local
        DocumentRoot /var/www/fail2stat
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/fail2stat>
                Options Indexes FollowSymLinks MultiViews
                #AllowOverride None
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
</VirtualHost>

Of course, I would recommend to run the site with https. Manuals for doing this can easily be found elsewhere.

The ServerName depends on whether you want to access the server from the internet or not. If so, you have to take care of DNS settings as well which is beyond the scope of this article.

Make sure that the DocumenRoot path exists and is writeble by the webalizer user.

Finally, activate the new site by executing with root privileges

a2ensite /etc/apache2/sites-available/fail2stat.conf
systemctl restart apache2

In the last line reload would be enough if the webserver is already running.

webalizer configuration

I assume you have webalizer already running as a cronjob. In this case just copy the configuration

cp /etc/webalizer/webalizer.conf /etc/webalizer/fail2stat.conf

and edit it to match the following lines:

OutputDir /var/www/fail2stat
CacheIPs yes
CacheTTL 30

The OutputDir must match the DocumentRoot of the apache configuration. The last two lines make sure that unresolvable IPs are not tried to be resolved all the time, the next time only after 30 days. This is useful because a lot of the devices trying to break into your system are not resolvable.

Test your configuration by webalizer -c /etc/webalizer/fail2stat.conf, maybe with root privileges, depending on your webalizer installation.

That's It

Now you should be able to see some useful statistics of fail2ban visiting http://yourserver.local.