Recently one of our client server was subjected to DDOS attack. We use Nginx’s Limit Req Module and fail2ban together to thwart this attack.
Installing fail2ban
On Ubuntu/Debian, just run…
apt-get install fail2ban
Configuration
There are 2 parts. First, we need to configure nginx to limit number of requests for IP addresses. Nginx will log info about banned IP into error log. fail2ban will parse nginx error log and ban offending IP addresses.
Nginx configuration
Please follow this post for nginx config part.
fail2ban Configuration
filter config
Create a nginx filter file:
vim /etc/fail2ban/filter.d/nginx-req-limit.conf
Add following content in it:
# Fail2Ban configuration file # # supports: ngx_http_limit_req_module module [Definition] failregex = limiting requests, excess:.* by zone.*client: <HOST> # Option: ignoreregex # Notes.: regex to ignore. If this regex matches, the line is ignored. # Values: TEXT # ignoreregex =
jail config
Create a new jail config in:
vim /etc/fail2ban/jail.local
If you don’t see jail.local, simply run:
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Add following towards end:
[nginx-req-limit] enabled = true filter = nginx-req-limit action = iptables-multiport[name=ReqLimit, port="http,https", protocol=tcp] logpath = /var/log/nginx/*error.log findtime = 600 bantime = 7200 maxretry = 10
findtime and maxretry values are important. Together, they decides how often offending IP’s gets banned. If you make these values smaller, IP’s will get banned more often. Tweak as per your need.
After saving both config files, restart fail2ban using:
service fail2ban restart
Testing
Before you exit from shell, it’s better to make sure if fail2ban is working.
fail2ban logs
You can monitor fail2ban log file:
tail -f /var/log/fail2ban.log
You will see lines like below:
2014-04-28 14:16:02,840 fail2ban.actions: WARNING [nginx-req-limit] Ban 95.211.117.202 2014-04-28 14:16:02,848 fail2ban.actions: WARNING [nginx-req-limit] Ban 78.187.45.204 2014-04-28 14:16:03,857 fail2ban.actions: WARNING [nginx-req-limit] 78.187.45.204 already banned 2014-04-28 14:17:36,952 fail2ban.actions: WARNING [nginx-req-limit] Ban 91.216.201.114
If you don’t see anything that means either misconfiguration or nothing to worry at all. If you think there is something to worry, jump to debugging section below.
fail2ban-client
You can also use fail2ban-client to find out status of a particular jail using following command:
fail2ban-client status nginx-req-limit
This will show:
Status for the jail: nginx-req-limit |- filter | |- File list: /var/log/nginx/test.com.error.log /var/log/nginx/example.com.error.log | |- Currently failed: 6 | `- Total failed: 389 `- action |- Currently banned: 3 | `- IP list: 95.211.117.202 78.187.45.204 91.216.201.114 `- Total banned: 3
As you can see there are 3 IP’s in jail.
Debugging
If things are not working as expected, you can debug fail2ban config.
Check debug output
Run following command to see config used by fail2ban-server:
fail2ban-client -d
Debug filter
Run following command to see if fail2ban filter works for a particular log file:
fail2ban-regex /var/log/nginx/example.com.error.log /etc/fail2ban/filter.d/nginx-req-limit.conf
Output will contain something like following (towards end):
Success, the total number of match is 861
If there are zero match then there could be an issue with regex filter.