rsync: synch both boot and data volumes.

If you have mission critical servers, you know that a backup is essential. I do not want my mission critical web server to go down. Along comes rsync.

How to set up rsync to synch my boot drive (and make it bootable) as well as synch my data files (html/php files):
First, you will to set up passwordless ssh between your servers. This is a good tutorial, except make sure step #5 is actually: /var/root/.ssh/cat >> authorized_keys2 . Now you are ready to send data from your main server to the backup(s).

I have a daily cron job to run my data rsync script.
The script is: time sudo rsync -a -vv -z -e ssh "/Volumes/www" "[email protected]:/Volumes".
(where /Volumes/www is the location of all my data files and is the IP of my backup web server)

Boot drive (that is bootable)
I have a weekly (I chose weekly because if I apply an update to something that proves not to work, I know I have some flexibility) (and yes I always apply updates to a test server first) cron job to run my boot script.
The script is:time sudo rsync -a --exclude /dev/* --exclude /afs/* --exclude /private/tmp/* --exclude /Network/* --exclude /Volumes/* --exclude /automount/* --exclude /private/var/run/* -vv -e ssh "/." [email protected]:/Volumes/BackUp --delete ; sudo rsync -qe ssh'`sudo bless -folder /Volumes/BackUp/System/Library/CoreServices`' .
(where /Volumes/BackUp is the location of all my boot(System) files and is the IP of my backup web server)

Next up: IP Failover! (another day)

Using rsync really cuts down on my worry..well a little, and also helps me keep all my servers synched to the primary server. Enjoy, it is time we started making these servers work for us, not the other way around!

why I use Mac OS X client on my Xserve for web serving needs.

I was recently asked to defend my decision to run OS X Client on my Xserve and I thought others might like to know why as well.

  • Changing the IP. I have written about this before. I have a mission critical web server. I want to upgrade the OS or the hardware, so for maximum up time, I configure my new server with a temporary IP, then when everything is ready and tested, I shutdown the mission critical server, change the IP on the new server, and everything and everyone is happy. Less than a minute of downtime. I am sure OS X Server is nice for people who do not have mission critical servers, but look in the mac-osx-server listserv archives, and you will see changing the IP is not as easy as Apple would like to think. (No, not even the changeip command works correctly. How am I supposed to upgrade any hardware or software when I need to test it first before I put it in production when I cannot use a temporary IP? I have tried changing the IP in Mac OS X Server 10, 10.1, 10.2, and 10.3 without success. Come on.
  • Industrial Strength Web Serving. (also known as web service without a smile) After reading Apache, The Definitive Guide, I had to wonder if Apple had ever heard of it. Apache is great, but the configuration on Mac OS X Server was a mess. (Mac OS X Server 10.3 made great strides to clean it up, but it still left more to desire).
  • Industrial Strength Performance Cache. Apple includes something called a “performance cache” but if you use the performance cache, it breaks some auth directives. (the ability to allow or deny by IP) Why would I ever need to do that for an intranet web site? I could live without the performance cache, but if you advertise something, and that you follow standards, don’t tell me I have to choose between the hyped up performance cache and Apache standards.
  • SSL certificates. Apple has a great GUI for adding SSL certificates, I will give them that, but please…why can I not change the file name of the SSL certificates, private key, etc without changing the SSL file names in every single vhost file…especially when 9/10 sites do not even use SSL? Yes, all the SSL directives are in each vhost conf file even though they do not use SSL. So I am telling you, if you decide to name your SSL cert file something other than what Apple has programmed (and Apple gives you an option to save the SSL cert anywhere) that Apache will not start because of syntax errors in each vhost conf? Yes. Should I be mad? Yes, I think so.
  • ErrorDocument. If you are like me, I have a script that depending on the error, an error specific page is created to let the users know exactly what the problem is. Slick, yes. Works in Mac OS X Server? No. What, but being able to define multiple error documents is very common and supported in Apache’s generic config. In the Server Admin GUI, Apple only put one field: Error Page. Perhaps that is acceptable for little web sites or for people who do nt care to differentiate errors to users, but I need it. I thought this would be easy, just add in the lines: ErrorDocument 404 error.php?404, ErrorDocument 500 error.php?500, etc right in the conf file that Server Admin creates for each vhost. Well, one slight change in the GUI to something completely unrelated (I changed the frequency of my log rolling) and my four or five ErrorDocument lines in the conf disappeared. What? My solution? Create another conf file that had the ErrorDocument directves in them for each site. Not acceptable. How many conf files must I have? A work around for a work around for a work around. This did not make me happy, especially when I went to the Apache Con 2003 and one of the speakers warned about performance hits when using multiple conf files.
  • Firewall GUI. What was not worthy of my complaints in Mac OS X Server 10.0-10.2 is worthy of a complaint in Mac OS X Server 10.3. There are many preset options to configure…the standard port 80 and 443 for web, port 22 for ssh, etc, but wow, try creating a rule for something like Retrospect (and why Retrospect was not included but Timbuktu was…). The ease of use really went south in Mac OS X Server 10.3. The old Firewall GUI was much easier to use. On principle, I have to complain about this because Apple should be trying to simply GUIs, not making them more complicated. (in my opinion) The Firewall GUI does not make me extremely upset like the web serving woes, but it adds to the fire.

In conclusion, the Xserve, is a really great piece of hardware and I am really excited about it, but the operating system that Apple has chosen to put on it and call “Industrial Strength” is not industrial strength for my mission critcal servers. If I have to give up my ability to use hwmond and watchdog (which I almost have working) I will. I have never had a rack mounted server with SMART drives, and fancy notification abilities so I will live. I am tired of hearing “but it is fixed in the next update” because I have wasted so much time reinstalling the next version of Mac OS X Server only to find other major errors which just create work arounds. I like Mac OS X Client as my web serving OS because Apple has left the conf files alone. No GUIs to mess with them. All my Apache directives work, and I can change my IP on the fly. I have sent Apple feedback, but as you can see, I am using Mac OS X Client on my Xserve for my web serving needs.

redirecting to

So you have a web server and one/some domain names. You want to have all requests to to go to most companies have this redirect in place (Apple is an example). I was stumped when I first set out to do this…so here is what I learned from our friends on the Apache mailing list.

Redirect permanent /

DocumentRoot /Volumes/www/

This set up in Apache will then redirect all requests to
You cannot beat mailing lists’ support!
[tags]apache, vhosts, redirect[/tags]

what's on my web server?

It is time to build another web server, and this time I will write about what is on it. I have built this configuration on the original iMac all the way up to the latest Xserve with success, so it is possible to make a cheap web server out of an older computer. To further save money, you could use all the apps with a different OS like FreeBSD, Darwin, NetBSD, OpenBSD, etc (I like the BSDs) which are free.

Tiger Server 10.4 Makes Me Growl

Well it is WWDC time and Apple has released information about my friend and your’s Mac OS X Server, the new improved version. We finally get mobile home directories, I am really excited about that, but then Apple talks about “Certificate management”….gee I wonder if you can rename the cert files (when you serve more than one site) and have apache start? That might be too much to ask, but I am glad Apple thinks it is something to mention. I also wonder how they are going to address the change ip issue. Time will tell, but I will not hold my breath. I would love to go somewhere, most likely a school district, set up Mac OS X Server to serve exactly like Apple thinks it should serve and have everything just work. I know it can, but if you try anything different…you will find problems. (see past posts) Mac OS X Server scares me for serveral reasons, but none more than the idea of UNIX admins looking at Apple’s awesome hardware and then being completely disappointed with the OS when trying to customize it. (you know, like adding more than one ErrorDocuments for a custom error reporting script…who would ever want that?) I can deal with Mac OS X Server issues…but I do not want to see other’s put off by Apple’s weird configurations. How can users talk to developers…and have the developers actually respond to the user’s issues? I have had engineers duplicate problems and release after release the issues still remain. Thoughts? We now have the hardware that is attracting high end users, let’s give them the software too!

Mac OS X Server, Apple, A Truce Please?

Well…one look at the “Safari cannot connect to localhost because your computer is connected to the internet” message I received yesterday while trying to view my web site while on the train and I about lost it. After I hacked the apache conf, that was enough for me to put up the white flag when dealing with Mac OS X Server. Apple needs to hire me or some other apache/mac based person to tweak the apache conf in Mac OS X Server. I am so frustrated at this point I am going to throw Mac OS X client on my Xserve so I can get away from Apple’s “weird trying to be helpful GUI”. I have written to the engineers, had them duplicate the problem, and then not fix it.
I do want to use Mac OS X Server, but not in the state it is now. So close Apple…so close, just hire me and I will help you obtain “Industrial Strength” status with web serving. 😉

Little Tech Reminders

I often perform the same tasks over and over and yet forget some little thing. This post is a reminder for me.

General Apple hints.

Mac OS X Server hints, aka Industrial Strength hints.

  • Don’t change Mac OS X Server’s IP 😉
  • When installing SSL certs on OS X Server (at least in 10.3 and below), if you change the cert file location from anything but the default, you will also need to change it in all your sites or Mac OS X Server will not be able to restart apache.
  • Apple: “Well Known” TCP and UDP Ports Used By Apple Software Products
  • Server Monitor not working after a clone? Make sure it is by adding: hwmond:respawn:/usr/sbin/hwmond # Hardware Monitor daemon to /etc/watchdog.conf

Tutorials and general resources.

Web development hints.

Changing the IP On Mac OS X Server, Lessons Learned

Is it common to change a server’s IP? I suppose yes and no. Yes, say you decide to move to a data center or your office moves or you want to configure a server to replace another server already using the IP you want for DNS reasons or you just need to reorganize your subnet. Yes, a change in IP is possible. Apple, thanks for figuring that out. I have been using Mac OS X Server since it first appeared on the scene and the sight of a “wheel” user really jarred me. Until now (Mac OS X Server 10.3.x) changing the IP was difficult at best. I called myself a “professional Mac OS X Server installer” for awhile because of needing to change the IP.

Every new version of Mac OS X Server promised ways of changing the IP with ease. Somewhere along the line, I think with Mac OS X Server 10.2 Apple even included an option in the install to use a temporary IP. Hmm, last week when I configured my Xserve (many times) the option was no longer available. Darn…it worked so well! (not) I had hopes for the new “changeip” (man changeip on OS X Server for more information) command, but like all other attempts to change the IP on my Mac OS X Server it failed, and failed miserably.

I do not really understand what the problem is. I have several Mac OS X clients that run web servers and other server processes and when I change that IP, the OS does not think anything of it. Mac OS X Server however is “industrial strength” and therefore has many more complicated issues. In theory you are can log in as root, type in a command like: “changeip oldhost newhost” to change the IP from to I was so sure Apple would not burn me that I spent a day configuring my Xserve and then finished the configuration over Remote Desktop at home so I could come in, log in in single user mode and run the changip command.

I walked in my office, happily rebooted the server in single user mode, typed the command in, and received a NetInfo error. No worries, the error code was no where to be found in Apple’s Knowledge Base(No the link is not incorrect, that is just my way of giving you first hand experience of what I got) NetInfo…another friend of mine. Ok, I thought, maybe something weird happened, I will just restart. Nope, same thing. Ok, so I booted up normally and logged in as root. Bingo! The command did it’s thing…or did it? I could now ping the server on the new IP. I was completely amazed, but with all my history with Mac OS X Server, I had to see it in the GUI to believe it.

I opened System Preferences>Network Settings. The old IP was still in the GUI. Ok…maybe it just needed a restart. Restarted, still the old IP remained. I changed it in the GUI, and the even gave it a restart. Why would I care about the GUI when I could ping? Mac OS X Server and I go way back. Guess what? None of my sites work. Guess what? All the configs come back fine, no errors. Guess what? You still cannot change the IP on Mac OS X Server without some pain. Thank goodness for Carbon Copy Cloner. I ended up just cloning the old server drive and putting that on the new Xserve. So close Apple, so close!

My Friend and Your's: Mac OS X Server

Installing and configuring Mac OS X Server is like being Larry David on Curb Your Enthusiasm. Larry has a great life, but there are always problems in the details…and the problems are unbelievable unless you see them yourself. I am still in pain from the install, the reinstall, the clone, and now onto the second clone. Apple: we love the Xserve. We love Mac OS X. But how did you screw up your “industrial strength” server OS? Let us modify the config files without worrying about the GUI eating our changes! I know I am very fortunate to not have to work with some other server OSes, but please listen to the users and let us help you build a truly “industrial strength” server. I would make this posting more humorous, but until I get a good install working, I will remain…not as happy as I could be.