Ban IP

Blacklist IP addresses on multiple hosts

© 2014-2016 Dennis Leeuw dleeuw at made-it dot com
License: GPLv2 or later


    1. Introduction
      1. The solution
    2. Install and setup
      1. The code
      2. Install
      3. Configuration
      4. Testing the setup
    3. Intergrate into fail2ban
    4. Distributed use
      1. lighttpd setup
      2. test the setup
    5. Some important notes


Most people will probably be using fail2ban or DenyHosts to protect their machines against brute force password attacks. After a certain amount of failures the offending IP is blocked in the firewall, so no further tries can be made. If you have more then one server tied directly to the Internet, the same tools are probably running on all these machines and after a certain amount of time all will be blocking the same IP offending address. In the meantime however a distributed attack across all your machines gives the bad guys (I assume guyes here, but it could be girls also) more accounts to test, since they distribute there database across your hosts assuming you are using one account database for all you hosts. So if you have set the amount of failures to 3 and have 10 machines they could try 30 accounts with passwords before getting blocked. So what I wanted was that if a single machine detects an attempt to break in it will block the IP address on all machines that I manage and that is what the orginal ban-ip stuff was all about.

In the meantime OpenSSH removed support for hosts.allow and hosts.deny files making DenyHosts absolete. People trying to break also got smarter and started testing account/password combinations once every half our, making most of the detection tools not notice them. And the last trigger to completely rewrite ban-ip was the fact that fail2ban has no whitelist support. Not having a whitelist means I have to enable IP addresses forever since people with just a couple of public/private-key pairs are blocked really fast when they have just three changes of doing it right.

The world is not ideal, so I had to figure out a way to get arround all those items. The first thing I was pointed to on the fail2ban mailinglist is this: That kills part of the problem. But I was not happy. It all seemed too difficult, and difficult things tent to break. So I looked again at my ban-ip stuff and what is now presented here as ban-ip is a total rewrite of what I had back then. The solution is now a independant daemon that maintain a backlist and a whitelist, a commandline tool to control the daemon, some filter and action files for fail2ban, and a PHP-script for the distribution of offending IPs. Enjoy!

The solution

We want to detect brute force attacks, but allow brute force SSH keys. With the above described problems we need to make it very unattractive for people trying to break in. So let's say we will block an IP address as soon as we detect 5 failed log in attempts in 12 hours. This way testing every half our is detected. So it effectively means that you can get away with it when you try to log in on our systems once every 2 hours (just a little more). I think they will soon look for another target to attack. The downside of this setting is that regular users will probably make typos in a worse rate then that, and esp. when they also use public/private keys they will be blocked within these 12 hours, so we need a whitelist to prevent them from being blocked. I do not know about you, but I do not want to maintain a whitelist for all the external IP addresses of our users, so it needs to be automatically maintained list.

The solution is simple. As soon as a user logs in he or she is an authorized user and thus the IP address is authorized. This IP address is then added to a whitelist with a time stamp. After that a whitelisted IP adres is never added to the blacklist. As soon as 36 hours are passed the IP address is removed from the whitelist. But everytime a user logs in from that same address the timestamp is reset in de database. Regular users with daily log ins will always be on the whitelist and will thus never get blocked. Those that are less frequent should just type their password correctly within 5 tries (within a 12 hour time span)... that seems reasonable to me.

Install and setup

The code

Released versions:
26 jan 2016: 0.8.2
25 jan 2016: 0.8.1
21 jan 2016: 0.8


All tools expect the ban-ip command to be available as /opt/ban-ip/ban-ip, since it is all scripting you can easily adjust it, but for now it is assumed that the provided tar-archive is extracted in /opt and that a symbolic link is present from the current version to ban-ip.

Example for version 0.8:

cd /opt
tar zxvf /path/to/downloaded/ban-ip-0.8.tar.gz
ln -s ban-ip-0.8 ban-ip

Create a /etc/ban-ip directory and copy the configuration file (ban-ip.conf) into it:

mkdir /etc/ban-ip
cp /opt/ban-ip/ban-ip.conf /etc/ban-ip

If you want to make use of the distribted ban functionality copy the ban-ip.php to the root of your webserver. More details are provided in the chapter called "Distributed use".


The configuration file is extensively documented, for the configuration options we refer to the /etc/ban-ip/ban-ip.conf file.

Testing the setup

NOTE: Currently ban-ip only uses ipset to maintain blocklists. It can however easily be expanded to support e.g. plain iptables, but I haven't found the time to implement it.

The ban-ip tool has a couple of functions:

Tie the ipset table to iptables
To make sure iptables uses our table use:
ban-ip link
Add an IP address to the blacklist table
block an ip address:
ban-ip ban
Delete an IP address from the blacklist table
remove a blocked ip address:
ban-ip unban
Add an IP address to the whitelist table
whitelist an ip address:
ban-ip whitelist
Delete an IP address from the whitelist table
remove a whitelisted ip address:
ban-ip unwhitelist
Show the current members of a list. blacklist or whitelist can be added to the command to show a certain list. Without any name the blacklist is shown.
ban-ip show
Start and stop the ban-ipd
For this the script has a start, stop and a restart function. Create the appropriate symlinks to the script from the different runlevels and your blocklist will be preserved over a reboot. It is important that this script is started before your fail2ban server starts, since the daemon is responsible for setting up the tables. Also make sure it intergrates correctly with your already present other firewall rules

To start using ban-ip start the daemon:

/opt/ban-ip/ban-ip start
and make sure that it runs:
/opt/ban-ip/ban-ip status
Daemon: running

After starting the daemon have an empty table ready to be filled with IP addresses that you want to block. Of course they are of no use if iptables is not looking at them, so create the link between iptables and ipset by running:

/opt/ban-ip/ban-ip link
You could test the setup by adding an IP as described above and using ban-ip show to see that the IP is added. Note that you are doing this on a "life" system, so if you pick your own IP address and are testing over an SSH-link, you ... right *click*...

This is now a working system in which you can manually ban/unban and whitelist/unwhitelist IP addresses. Now every kind of tool you want to use can be integrated with this to call the /opt/ban-ip/ban-ip script to ban and unban IP addresses. In the next chapter we will describe how we did it using the fail2ban tools.

Intergrate into fail2ban

fail2ban is a really good tool to watch your logfiles and tell some external tool about the problems it found. To make fail2ban do what we want it to do we create a /etc/fail2ban/jail.local file:

ignoreip =
bantime = 1
# findtime: 12 hours
findtime  = 43200
maxretry = 5
action = ban-ip

enabled = true

enabled = true

enabled = true
So we let fail2ban check our SSH-server and generic log ins that are done through services that use PAM. The fail2ban server will check for a period of 12 hours to find any wrong doings. As soon as 5 entries from the same source are found the address within that 12 hour frame the IP address is added to blocklist for 1 second.

Now we need to teach fail2ban that there is also something like "good users". We do this by creating a filter that reads the SSH log file and filters out succesful ssh log ins. We stored this filter as /etc/fail2ban/filter.d/ssh-whitelist.local:


before = common.conf


_daemon = sshd

failregex = ^%(__prefix_line)sAccepted \S+ for \S+ from \s.*$

ignoreregex =
As you can see the "failregex" is actually the Accepted rule, so a succesful log in.

Next we extend /etc/fail2ban/jail.local with an action plan:

enabled = true
maxretry = 1
logpath = %(sshd_log)s
filter = ssh-whitelist
action = ban-ip-whitelist
What we tell fail2ban is that everytime (maxretry=1) it detects an action as defined in the ssh-whitelist filter it should run the ban-ip-whitelist action.

Next we need to create the action that we called in the previous step. Create /etc/fail2ban/action.d/ban-ip-whitelist.local and fill it with:

actionstart =
actionstop =
actioncheck =
actionban = /opt/ban-ip/ban-ip whitelist 
actionunban =

name = default
The main thing it does is tell fail2ban to do nothing when starting, stopping, checking and unbanning. Only when it is called as ban-action we call an external script that blocks the IP address.

The same we need to do for the regular blocking of IP addresses. So we add an additional action as /etc/fail2ban/action.d/ban-ip.local that looks like this: [INCLUDES] [Definition] actionstart = actionstop = actionban = /opt/ban-ip/ban-ip ban actionunban = [Init] name = default So here also fail2ban is mostly doing nothing.

When we start fail2ban it will do what we want it to do: Detect succesful log ins and tell ban-ip about it and detect failed log ins and after 5 such attempts in 36 hours it also tells ban-ip about it. Sounds like a prefect detection system.

Distributed use

The easiest way to distribute a blacklist or an offending IP address is probably by using a well tested protcol like HTTP. No need to reinvent the wheel.

Within the code package there is a PHP-file which provides a couple of functions which makes it possible to receive an detected IP address to be blocked. To be precise it can add and delete IPs and it can list the the blacklist. The ban-ipd uses curl to send the IP address to a webserver providing the PHP-file and the PHP-program calls /opt/ban-ip/ban-ip with a command to make it all happen. To be able to call the ban-ip command from a webserver, the user under which the webserver is running should be able to call the ban-ip script. To accomplish this one could use sudo (or something else):

lighttpd	localhost=(ALL)			NOPASSWD: /opt/ban-ip/ban-ip *
we assume lighttpd here running under its own username.

Check that sudo is working correctly by using:

sudo -u lighttpd sudo /opt/ban-ip/ban-ip show

The path to ban-ip is hardcoded in the PHP file, so if you installed the script in another place then change it in the above sudoers example and also in the PHP code.

The /etc/ban-ip/ban-ip.conf file contains 2 settings for our destributed system that need to be correct. First there is the notify_array that should hold the IP-addresses or names of the servers to contact and second notify_php_script is the complete path and filename to the PHP-script on the remote server. So something like this might work:


The ban-ip.php file can be used on a single host that maintains a database for all your hosts, or you could add an HTTP server to all your servers and add this PHP-script to all of them. We assume you will also use lighttpd and that the PHP-script is installed in /var/www/lighttpd. If you use another HTTP-server skip this section.

lighttpd setup

Install lighttpd, spawn-cgi, php-fpm. For the different distributions we have:
  Package php-fpm.conf dir www.conf dir
Debian php5-fpm /etc/php5/fpm /etc/php5/fpm/pool.d/
Debian spawn-cgi    
Debian lighttpd    
Red Hat php-fpm /etc /etc/php-fpm.d
Red Hat spawn-cgi    
Red Hat lighttpd    
Adjust in php-fpm.conf the daemonize variable and set it to yes. Adjust the www.conf file and set:

listen =
listen.allowed_clients =

On Debian based systems make sure /etc/php5/cli/php.ini and on Red Hat based systems make sure /etc/php.d/lighttpd.ini that:

cgi.fix_pathinfo = 1
is set.

lighttpd provides a /etc/lighttpd directory with the configuration, we are not going to use the distro specific setup, but create our own, just to keep this document simple. Create in the configuration directory a lighttpd.conf with the following content:

server.modules = (

server.username        = "lighttpd" # www-data on Debian
server.groupname       = "lighttpd" # www-data on Debian
server.port            = 6543
server.use-ipv6        = "disable"
server.max-connections = 10 # amount of servers that can connect at once

# Some variables we could use
var.www_root_dir     = "/var/www/lighttpd"
var.log_root_dir     = "/var/log/lighttpd"
var.tmp_upload_dir   = "/var/tmp/lighttpd"

# File locations
server.upload-dirs   = ( tmp_upload_dir )
server.document-root = www_root_dir
server.errorlog      = log_root_dir + "/error.log"      = "/var/run/"

# Content handling
index-file.names               = ( "index.php", "index.html", "index.htm" )
url.access-deny                = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".scgi" )
server.follow-symlink          = "disable"
server.dir-listing             = "disable"

# For 2.6 kernels and up
server.event-handler = "linux-sysepoll"

# linux-sendfile - for small files.
# writev         - for large files = "linux-sendfile"

# Found on CentOS 6:
## disable range requests for pdf files
## workaround for a bug in the Acrobat Reader plugin.
$HTTP["url"] =~ "\.pdf$" {
  server.range-requests = "disable"

# Pick one
# Debian:  include_shell "/usr/share/lighttpd/"
# Red Hat: include "conf.d/mime.conf"

server.modules += ( "mod_fastcgi" )
fastcgi.server = ( ".php" =>
                   ( "localhost" =>
                       "host" => "",
                       "port" => 6542
There are two things that you need to look into. The username and groupname that the server should run as, and the way the mime info is included. Adjust to your distribution and safe the file.

With this in place restart lighttpd and php-fpm or php5-fpm.

Create a test.php in /var/www/lighttpd and fill it with: <?php phpinfo(); ?>. Now you test your setup by accessing this file from your browser. If all is fine, remove test.php from the directory and you are ready to go.

test the setup

To control the black list use:

to add an IP to the blacklist, or:
to delete an IP address from the blacklist.

You can also list all IPs in the table by using:


Some important notes

Important NOTE: It is easy for a normal user to fool the fail2ban system by using something like the following on the command line:

logger -p auth.warning -t 'sshd[666]' 'Invalid user GuessMyName from'
logger -p auth.warning -t 'sshd[666]' 'Invalid user GuessMyName from'
logger -p auth.warning -t 'sshd[666]' 'Invalid user GuessMyName from'
This will block IP, and with a script a user can DDoS or completely block your system. On the other hand it is also a very simple tool to test your configuration. To prevent the lock-out of your system make sure you have a ignoreip list in your jail.local that always allows remote access from a couple of sysadmin machines.

Another important NOTE is that the current limit on an ipset table is 65535 entries, which you can change using the maxelem option when creating the table.