OMNINOGGIN

Every time I search the web for information on how to block spam bots, scrapers, and harvesters, I always see an Apache .htaccess file or some code to dump into httpd.conf to achieve this. I’m a bit against using this method for blocking evil bots. I do respect Apache for being a flexible & modular web server (that’s why I still use it), but I do not have much to boast about Apache’s speed and efficiency.

To achieve the same purpose on my server with greater efficiency, I made use of my Varnish reverse proxy configurations (located under /usr/local/etc/varnish/default.vcl).

In this post, I will only be discussing about vcl_recv subroutine, which gets called when a client request is received.
Read on…