I started by following the Open Addict tutorial section “Configuring Mail Services”. After the step “Edit the [hostname].mc file, add these lines, I put mine above DAEMON_OPTIONS,” I stopped following the tutorial and did the following:

1. Add these lines into my [hostname].mc file (above DAEMON_OPTIONS). Also note that “smtp.sbc.mail.yahoo4.akadns.net” is SBC Yahoo’s mail proxy server.

   define(`confAUTH_OPTIONS', `A')dnl
   define(`confAUTH_MECHANISMS', `PLAIN LOGIN')dnl
   TRUST_AUTH_MECH(`PLAIN LOGIN')dnl
   FEATURE(`authinfo',`hash /etc/mail/authinfo')dnl
   define(`SMART_HOST',`smtp.sbc.mail.yahoo4.akadns.net')dnl

2. Create the file /etc/mail/authinfo and put the following into it (all in one line) and save (also make sure you replace <your-username> and <your-password> with your real username and password):

   AuthInfo:smtp.sbc.mail.yahoo4.akadns.net
   "U:<your-username>@sbcglobal.net"
   "I:<your-username>@sbcglobal.net"
   "R:sbcglobal.net" "P:<your-password>" "M:PLAIN"

3. Run the following command:

   makemap hash /etc/mail/authinfo < /etc/mail/authinfo

4. Set sendmail to start (and listen) at boot, so add this to /etc/rc.conf:

   sendmail_enable="YES"

5. Create the file /etc/mail/local-host-names and insert your domain name(s) to accept mail for. Then, finish the sendmail re-build.

   touch /etc/mail/local-host-names
   make all install restart

6. Send a test email to yourself

   sendmail -f "webmaster@your-domain.com" -F "webmaster"

   From: Webmaster <webmaster@your-domain.com>
   To: your.other@email.com
   Subject: Hello World!
   Hello World!
   .

This should work but in case it doesn’t, there is some good troubleshooting information on DSL Reports. It took me 5 hours to figure this out so good luck!

Since all you have to do to update your ports is run the command:

cvsup -g -L 2 /root/ports-supfile

Just stick that command in your crontab and make it run every midnight.

crontab -e # This opens the crontab in the vi editor.

Add this line to the crontab:

0 0 * * * /usr/local/bin/cvsup -g -L 2 /root/ports-supfile

Shift+zz to save and you are done.

This it one of the reasons why I love Unix. With little thought, everything is script-able and customizable.

1. Installing OS
2. Setting Up Apache, MySQL, and Other Services
3. Migrating Wordpress from WinXP to FreeBSD
4. Optimizing Apache
5. Optimizing MySQL
6. Optimizing PHP
7. Proxy Caching
8. Optimizing Wordpress with WP-Cache
9. Keeping Your FreeBSD Ports Up-to-Date Effortlessly
10. Setting Up Sendmail on FreeBSD 6.2

Once installed, Varnish will keep a cache of all objects requested by internet users (e.g. post-generated PHP pages, CSS, javascripts, images) with the goal of off-loading some work from your web server (remember: we won’t want big Apache to do the work only if it has to). Also Varnish takes full advantage of the OS’s virtual memory and advanced I/O features on FreeBSD 6.x making it the optimal choice for my setup.

There were many confusing instructions on the web about how to configure Varnish. Here are the steps I took to setting up Varnish for a signal machine running both Varnish and the web server:

1) Port install Varnish.

cd /usr/ports/www/varnish/; make install clean

2) Change Apache’s port to 8080 by modifying this line in /usr/local/etc/apache/httpd.conf

Listen 8080

3) Restart Apache

/usr/local/etc/rc.d/apache22 restart

4) Start the Varnish daemon. Substitute the IP below with your machine’s internal network IP address. If you are hosting at home, it is most likely 192.168.0.* but you can run ifconfig to find out what it is.

varnishd -a <your-machines-internal-ip-address>:80 -b localhost:8080

That’s it! From now, all requests on port 80 should now be routed through the Varnish proxy cache. A proxy cache miss will seemlessly make Apache calls to serve the internet user without the user ever knowing it! A proxy cache hit would not access Apache (and therefore not access MySQL) lessening your machine CPU usage and increasing the speed & efficiency of serving objects to user.

When a PHP script is executed, the PHP interpreter will spend some time interpreting the script then compile the interpretations into opcodes for execution. eAccelerator will precompile your PHP code into ready executable opcodes and manage that opcode cache for you. If your PHP script does not change, Apache will directly call the precompiled opcodes (saving interpretation and compilation time).

This is what I did to set it up:

1) Port install eAccelerator

cd /usr/ports/www/eaccelerator; make install clean

2) Inject these lines into /usr/local/etc/php.ini

; eAccelerator Stuff
extension="eaccelerator.so"
eaccelerator.shm_size="16"
eaccelerator.cache_dir="/tmp/eaccelerator"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="1"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="1"
eaccelerator.compress_level="9"
eaccelerator.allowed_admin_path = "/usr/local/www/apache22/data"

3) Make cache directory writable

mkdir /tmp/eaccelerator
chmod 777 /tmp/eaccelerator

4) Restart Apache

/usr/local/etc/rc.d/apache22 restart

That’s it! This 10 minute optimization slashed my page generation time by 25%! But if that’s still not fast enough for you, I will show you how to do Proxy Caching next.

max_connections = 10
max_user_connections = 10

These numbers are a bit tricky to setup. The value depends on the ratio of dynamic requests and static requests. Dynamic requests are typically PHP pages that make MySQL queries and get regenerated every time you access a page. Static requests are typically files that are loaded/imported (e.g. *.js, *.css, *.jpg, *gif, etc) . The average ratio for this is usually 1 dynamic request for every 5 requests, but I recommend you using a log analyzing software to find the exact number.

I used Analog to help me find this number. After you’ve ran Analog on your /var/log/httpd-access.log file, you should get a report file. In the report file, find the section “File Type Report” and note the percent of calls made to the *.php file type. That is your ratio.

Multiply your ratio by the number of maximum connections you’ve configured for Apache. For example, if my max number of Apache connections is 27 (from previous post) and my ratio is 1/6, my MySQL max_connections should be CEILING(27 * 1/6) = 5. I multiplied that number by 2 for safety measures to get max_connections = 10.

You have to set max_user_connections = 10 too because sometimes MySQL thinks that Apache is just 1 user and you do not want Apache to be limited to only one connection.

max_allowed_packet=1M

I’ve set max_allowed_packet to 1M as a measure to prevent run-away queries. I’m pretty sure that I won’ be querying anything that will returning a data packet greater than 1M.

table_cache=100

The table cache setting helps speed up MySQL queries. The value set should be max_user_connections multiplied by the number of JOINs your heaviest MySQL query contains (In my case, 10 JOINs).

query-cache-type=1
query-cache-size=10M

This enables the query-cache feature. Wordpress doen’t need a big query cache so I’ve set mine at 10M.

After all this, restart MySQL and watch your Wordpress page generation time dip by 50%. Ooo sweet sweet reward. Next I will show you how to optimize PHP!

Preface

Before you do anything, I recommend you add the following lines of code into footer.php of your Wordpress blog:

<?php echo get_num_queries(); ?> queries
<?php timer_stop(1); ?> seconds

This will tell you the number of MySQL queries called and the number of seconds it took to generate the page you are on.

Once you’ve played a little bit with the timer_stop(1) function, you will notice that the first time generation of a page will take longer than successive tries. This is because Wordpress has an internal cache mechanism built in and it is on by default. If you are happy with the page generation time then you should just ignore my tutorial. To me, I cannot stand leaving anything unoptimized because why be wasteful when you don’t have to be? Also in case your blog gets dugg, you will have the pleasure of seeing that your blog can withstand hundreds of visitors without crashing like most others.

Optimizing Apache

For the record, my blog is hosted on a dedicated 2.0GHz P4 with 1GB of DDR RAM. This information (at least the memory part) is important to figuring out how to optimize your Apache. One of the most common reasons why blogs crash is because Apache is able to run loose and spawn an infinite number of httpd threads; thus, killing the memory and the machine. We need to tell Apache to STOP when we know we have reached our physical limitation. This is easily done by modifying your Apache configs in your httpd.conf file. Here is what I added to my httpd.conf:

MaxKeepAliveRequests 0
KeepAliveTimeout 0

KeepAlives is a feature that keeps a connection open for the duration specified by “KeepAliveTimeout”. When enabled this will alleviate the visitor of the overhead needed to create a new thread request if the next request occurs within the KeepAliveTimeout. You should turn this off because the overhead is minuscule and leaving it on will waste memory.

ServerLimit 27

This setting is important because it would stop Apache from spawning too many httpd threads and fill up all the memory. Once the memory is filled up, your system will automatically start using swap space. It is VERY BAD for a web server to be served from swap space! The setting above tells apache to not spawn more than 27 httpd threads. Here are the formulas to help you find out what your server limit should be.

If you are sharing your machine with anything else (e.g. MySQL database) you should use the formula below:

MaxClients = Total RAM / (2 * Apache Process Memory Size)

If you are using your machine solely for Apache you should use the following formula:

MaxClients = Total RAM / Apache Process Memory Size

My RAM is 1024MB and my Apache processes are about 19MB each. I also run MySQL on the same machine so using the first formula gives me ~27.

MinSpareServers 15
MaxSpareServers 20

These 2 settings mean just what they say. MinSpareServer specifies the minimum number of httpd threads to be left for spare usage at any given time. MaxSpareServer specifies the maximum. The trick is to adjust this to best suit your traffic. The general rule of thumb is that if Apache needs to spawn more than 4 threads at any given time then you should increase your MinSpareServers value. MaxSpareServers is used to keep spawned spare threads in check. For example, if visitors are actively using 5 of your servers, you will have 15 spare servers sitting in the background in case usage increases dramatically. When visitor usage increases to 8, Apache might increase the number of spare threads to 20. Once usage increases to 10, Apache will not spawn anymore spare threads.

MaxClients 27

If you know that your machine cannot handle 100 users then you should cap it to what you think it can handle. I’ve changed my max clients to 27 (1 per httpd thread).

ListenBacklog 27

This setting tells Apache to listen to the next 27 requests. If you know you cannot handle those extra requests it is better to deny the user right away instead of having them wait for a server timeout.

MaxRequestsPerChild 4000

Contrary to what this looks like, this is the number of requests the child handles until it dies. With the setting above, one httpd thread will handle 4000 requests then it will die. If you keep the original setting (0, unlimited), the child process will never die and it might cause memory leakage issues later on.

After saving your httpd.conf, restart your Apache server and the changes should take effect. Now, you can move onto the next step, MySQL optimization.

I previously made a an awfully painful choice to host this site on my WindowsXP machine using WAMP (FYI, WAMP = Windows, Apache, MySQL, PHP packaged into one install). WAMP and I had a 10-month-long love-hate relationship:

Month 1

- WAMP quickly attracted me with its sexy promises of hitting it off quickly.
- There was no need to set up a separate OS/machine and learn it.
- I loved the fact that it took me less than 30 minutes to get my server up.
- Website administration is a lot easier since it’s on the local machine.

Months 2 – 10

- I had to set up VNC, FTP, and SSH to be able to remotely administer my website if I am not home.
The longer Apache and MySQL processes stay alive, the more memory they hog (memory leaks). I had to keep restarting the services every day to maintain acceptable performance.
- Windows keeps crashing or auto-updating causing my server to restart without notice. I had to turn off auto-update. As for the random crashing, all I was able to do was hope that it won’t crash.
- WAMP started to get fat and lazy once I added more content into to the site. I had 8 second page generation times!
- When I tried to follow optimization guides for Apache, MySQL, and PHP, I found out that Apache packaged with WAMP had limited httpd.conf functionality. I tried MySQL optimization and it did not help the time much. Lastly, I could not find a PHP accelerator software for windows PHP5. This was the straw the broke the camel’s back.

As you can see, this story is like the story of meeting a supermodel girl. She might be fun in the beginning but after the first month, you find out that she’s actually a high maintenance airhead who doesn’t know how to do anything but strut her stuff. After 10 months I was ready to get myself a house wife that will cook, clean, and quickly serve pages.

How I Broke up with WAMP

Here is a general outline of the steps I took to migrate my site over from Windows XP WAMP platform to FreeBSD.

1. Upgrade Wordpress: Lazy me, I had an older version of Wordpress. I had to take this extra step to upgrade my Wordpress because the newer version will most likely make modifications to MySQL.
2. Export old MySQL data: You can either do a mysqldump or you can be chicken like me and use phpMyAdmin to export everything in my old MySQL server to a localhost.sql file.
3. Import data to the new MySQL server: On the command line of the FreeBSD machine I did:

   mysql -u root -p < localhost.sql

   Of course this did not work the first time. It might work for you the first try but in my case, I had an error about root not being able to access some of the tables and databases. I had to go and modify localhost.sql and add GRANT permissions for “root”@”localhost” to all of the tables & databases. After I did that, everything imported fine.

4. Migrating Wordpress: I copied all files over from the old machine and everything worked like a charm (Keep in mind that I kept my database users/passwords the same). Also, MAKE SURE YOU BACKUP EVERYTHING BEFORE YOU DO THIS!

I was FREE AT LAST!

Setup Useful Daemons

The only daemon I use is ntcd used for time synchronization. “net-snmp” will be paired with Cacti (a nice network graphic utility with a web interface) so it *could* be useful to you, but for the time being, it doesn’t do much for me. I will most likely revisit net-snmp & Cacti later as my site gets more popular.

Configuring Mail Services

I have to admit that I was not able to figure out how to configure mail services on this server. I’m pretty sure I followed all the steps correctly, so either something is wrong in the steps or I have an invisible firewall somewhere on my network that doesn’t allow me to send mail from my server. In any case, you might be better off looking for a different tutorial online about setting up mail services on FreeBSD. For you experts, please let me know if any of you have had experience setting up a mail service on FreeBSD or know of a reliable tutorial.
*Update* – 10/10/2007
I was able to successfully setup sendmail and wrote a quick guide about it, Setting up Sendmail on FreeBSD 6.2. Enjoy!

Configuring Web Services

Overall the Apache instructions are pretty good. Again I skipped the squirrelmail install since I was not able to get mail services to work.

I also ran into a minor problem because my Wordpress’s “Lazyest Gallery” plugin needs PHP GD2 Library to work. There are no instructions on how to do this in the tutorial or on the internet (based on the saying, “There is so much information out on the net that it is very hard to find the information you need.”) . After a couple trial-and-errors, this is what I did to get GD2 Library to work with PHP5:

cd /usr/ports/lang/php5-extensions/; make install clean

A customization screen will pop up with default extensions to install. I unchecked everything and then check the GD Library option. After that, I restarted Apache and it worked like magic. Also, if you ever need to come back and reconfigure php5-extension, all you have to do is the following:

cd /usr/ports/lang/php5-extensions/
make config # this will bring up the customization screen
make install clean

Configure System Graphing, powered by MySQL

I followed the MySQL instructions and it worked like a charm. Again I skipped Cacti setup for reasons I previously mentioned in the first paragraph.

Configure Bind

I skipped this step since I have a really nice router doing DNS resolution for me.

For the record, I have a Linksys WRT54G router with the DD-WRT firmware. There are many after-market router firmwares out there promising the world. I’ve tried a few high rated ones but in the end I chose DD-WRT for a couple important reasons.

1. It’s so stable that uptimes of greater than 3 months is common. The reason why it’s usually never more than 1 year is because I live in California. Damn that Arnold.
2. It has so many built-in features that you would otherwise have to setup on an independent Unix machine. Some uncommon things I’ve used it for are traffic tunneling, DNS resolution, dynamic DNS updating (works with Zonedit and other dynamic DNS services), QoS, and Samba mount.

Okay, I’m out of my soap box now.

Conclusion

Setting up Apache, MySQL, PHP was not as hard as I thought. Little did I know that the hardest part will come when I start optimizing the server.

I followed this really nice tutorial about installing a FreeBSD 6.2 OS. When you want to setup a server, it is typically best to perform a minimal installation (as shown how to do in the tutorial at Open Addict). I have to admit that there were a few typos in the tutorial that jump up and bit me later on in the process. I will try to point those out here since there aren’t many question & answers about this tutorial in the Open Addict forums.

All of the instructions are good up until the section called “Rebuild the World, and Recompile the Kernel”. Rebuilding the world is usually done if you want to upgrade your OS (kind of like a windows update, except a lot more involved than clicking “Install”). If you have just downloaded your FreeBSD OS, you probably have the latest version of the OS with the latest patch and you probably won’t have to rebuild the world. Even if you do not need to rebuild the world you should perform some of the steps shown in the tutorial just because it will save you time later on.

Installing Basic Packages

In the step below:

pkg_add -r bash; pkg_add -r cvsup-without-gui; pkg_add -r nano

I recommend omitting “pkg_add -r nano” because:
1. FreeBSD minimal install already comes with “vi” as a text editor.
2. “nano” is a crappy text editor (IMHO).
3. “vi” is the industry standard (I’ve used “vi” since college).
4. The longer you use “vi” the more powerful it becomes as you learn tricks to speed up your productivity.

So if you heavily use “vi” like me, I would recommend doing “pkg_add -r vim” instead. (FYI, vim = “VI-Improved”)

Setting Bash as the Default Shell

I like bash, so I recommend this step also.

Updating Source and Ports Tree

If you see you need to upgrade your FreeBSD then you will need to update your source tree. The tutorial said I should use RELENG as release tag. I’ve tried it and it doesn’t work. I had to use RELENG_6_2 to get my source tree updated. I found some helpful documentation and update servers for cvsup.

You should update your ports tree as it will be very useful later when you are setting up things like Apache, MySQL and other software.

Rebuilding the World

You may skip this step if you do not want to update your the FreeBSD with the latest patches. The tutorial is pretty good here except for one typo about “ZUES”. Just replace that with “ATHENA” and it will make a lot more sense. Also if you are not going to rebuild the world, it is good to do the last two steps:

pkg_add -r ncftp; pkg_add -r wget
adduser <yourname>

Conclusion

Now I have a rad OS, on to setting up services!