A big thank-you goes out to whoever launched the DoS attack against this website! My pile of HAProxy managed Raspberries worked great! I noticed this in my Munin graph. (Yes, I will show how to configure Munin in an upcoming post!) People get upset when they are attacked. I find that they make great learning opportunities!
This attack was repeated web requests for “POST /xmlrpc.php”, a fairly well-known attack for a WordPress website. These requests came in once per second and the HAProxy / Raspberry website config lasted through the storm!
One thing that did come to light was my logging. When I went to look at the client IP of the attacker, I realized that apache was logging the IP address of the proxy server! Ugh! So, I needed to fine-tune a few things. Let’s jump into setting up remoteip logging for apache.
HAProxy revisions
defaults option forwardfor
Add the forwardfor option to the default section of your haproxy.conf and restart.
Apache.conf revisions for forwarding
RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 10.0.0.203/8
RemoteIPTrustedProxy proxy.local
LogFormat "%v:%p %a %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%a %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
Edit the logging section of your apache2.conf file. I added the RemoteIP lines and I changed all the instances of %h to %a. Finally, you need to enable the remoteip module. In Raspbian’s version of apache, this is done by symlinking the module from the mods-available to mods-enabled:
cd /etc/apache2/mods-enabled ln -s ../mods-available/remoteip.load remoteip.load
and then restart apache. Now, I can see where my traffic (bad or good) is coming from and start banning some of these addresses.
Now, to spin up some salt scripts to reconfigure my web servers.
