So taking a page out of Anubis' book, I went and implemented my own little bot blocker. The idea is pretty simple. Each request gets checked for the presence of a cookie. If the cookie is set, the request is served as usual. If the cookie is missing, a simple HTML page with a button is shown. Real users are asked to click the button, get a cookie valid for 30 days and the page reloads, this time serving the original request. From then on they can browse the site as usual. But since bots don't click buttons (yet), they are stuck at the button page forever.
I was able to implement this whole mechanism with Apache's mod_rewrite module, which means no additional service (like Anubis) is needed. Each request is checked by Apache and since the bot check page is static, nearly no resources are needed.
It works really well. Can you spot when the system went online in the graph below?
