Block Unwanted Spam Bots Using Varnish VCL

User ImageThaya Kareeson
Popularity: 30%
Updated: Jun 18, 2008


Every time I search the web for information on how to block spam bots, scrapers, and harvesters, I always see an Apache .htaccess file or some code to dump into httpd.conf to achieve this. I’m a bit against using this method for blocking evil bots. I do respect Apache for being a flexible & modular web server (that’s why I still use it), but I do not have much to boast about Apache’s speed and efficiency.

To achieve the same purpose on my server with greater efficiency, I made use of my Varnish reverse proxy configurations (located under /usr/local/etc/varnish/default.vcl).

In this post, I will only be discussing about vcl_recv subroutine, which gets called when a client request is received.
Read on…


Creating a Staging WordPress Blog for Testing

User ImageThaya Kareeson
Popularity: 28%
Updated: Jun 24, 2008

Over the past few months, I’ve been meaning to create a staging WordPress blog that is an exact replica of my production OMNINOGGIN blog so I can test major feature changes before releasing them to my production site.  I have to admit that there are many other interesting things to spend time on (see also: Make Popularity Contest Work with WP-Super-Cache and NowThen Photo Display WordPress Plugin) so I have been lagging at getting this task done.  Fortunately the WordPress 2.5 released was enough to motivate me to get this done.  My goal in this post is to provide a step-by-step set of instructions (or checklist) for getting this task done.  I run Apache 2.2.8, MySQL 5.0.51a, and PHP 5.2.5 on a FreeBSD 7.0 machine that I have complete control over.  Keep in mind that these steps will vary depending on how your blog is configured.  It is a good checklist nonetheless so without further ado:

Read on…


Recapping: Setting up a FreeBSD 6.2 Web Server

User ImageThaya Kareeson
Popularity: 89%
Updated: Jun 18, 2008

I hope I can get some part-time consulting jobs to do this optimization for small businesses. All in all, it doesn’t seem too hard to do and I enjoyed doing it. If you run into a problem just google it for the answer. Anyway, here is the recap of the steps I took to set up my FreeBSD 6.2 Web Server.

  1. Installing OS
  2. Setting Up Apache, MySQL, and Other Services
  3. Migrating Wordpress from WinXP to FreeBSD
  4. Optimizing Apache
  5. Optimizing MySQL
  6. Optimizing PHP
  7. Proxy Caching
  8. Optimizing Wordpress with WP-Cache
  9. Keeping Your FreeBSD Ports Up-to-Date Effortlessly
  10. Setting Up Sendmail on FreeBSD 6.2
Rate this:
2.5

Setting up a FreeBSD 6.2 Web Server: Proxy Caching (Part 7)

User ImageThaya Kareeson
Popularity: 65%
Updated: Jun 18, 2008

Okay I lied, eAccelerator gives a pretty darn high ROI, but setting up a proxy cache gives a comparable or higher ROI. I chose to use Varnish as my proxy cache.

Once installed, Varnish will keep a cache of all objects requested by internet users (e.g. post-generated PHP pages, CSS, javascripts, images) with the goal of off-loading some work from your web server (remember: we won’t want big Apache to do the work only if it has to). Also Varnish takes full advantage of the OS’s virtual memory and advanced I/O features on FreeBSD 6.x making it the optimal choice for my setup.

There were many confusing instructions on the web about how to configure Varnish. Here are the steps I took to setting up Varnish for a signal machine running both Varnish and the web server: Read on…