Anomalous Anomaly

UPDATE (March 5, 2010): I’ve updated the article to reflect that I now keep my FreeBSD /boot on the ZFS partition.

You may remember my last article on a similar topic. At the time, I thought any more than three operating systems couldn’t be done on a Mac. But surprisingly, I’ve managed to install four operating systems with relative ease.

This time, I’ve got Mac OS X (10.6.2), Windows 7, Ubuntu Linux 9.10, and FreeBSD 8.0 installed.

Historically, there have been a couple problems with doing quad booting. Most operating systems (Linux and FreeBSD namely) don’t do well when booted directly from EFI, because this boot method skips Apple’s BIOS emulation. This prevents using graphics cards with hardware acceleration enabled, among other things. Being a game developer, this means that EFI-booting Linux and FreeBSD is not an option for me. The second problem is that the MBR/GPT hybrid only allows the MBR side to have at most 4 primary partitions and no extended/logical partitions. In the past, this has meant that things booted under BIOS emulation can only “see” the first 4 partitions unless they have built-in support for GPT.

In Linux’s case, it’s simple to get it to “see” past the 4th partition by compiling the kernel with GPT support. Linux’s boot process on a tri-boot or quad-boot mac goes something like this:

1. rEFIt switches from EFI-boot mode to BIOS emulation and starts GRUB, using the third partition’s boot sector.
2. GRUB initializes, and since GRUB only has support for the MBR, it can only see the first 4 partitions.
3. GRUB loads and boots the Linux kernel from the third on-disk partition (which is Linux’s /boot partition).
4. The Linux kernel initialises, loads its built-in GPT support and can see past the 4th partition.

So Linux has no problem getting past the MBR’s 4-partition limit. Now, how do we deal with FreeBSD? As you can see from the partition table below, FreeBSD’s partitions are past the MBR’s 4-partition limit (illustrated by the gray line).

So how do we boot FreeBSD without using a EFI-based boot? We upgrade GRUB to GRUB2. The newer GRUB2 has support for UFS2 (which is a filesystem FreeBSD’s /boot can use) as well as support for GPT. Ubuntu also has a patched GRUB2 which has support for ZFS. It’s certainly possible to compile their spin of GRUB2 on other Linux distributions, but it’s not exactly a cakewalk. I ended up compiling it myself so that I could use Fedora instead of Ubuntu, though.

All that’s needed to get GRUB to boot FreeBSD (in the case of the partition layout above) is to add this to Ubuntu’s /etc/grub.d/40_custom, and then run ‘update-grub’ as root:

menuentry "FreeBSD 8.0" {
	insmod zfs
	set root=(hd0,6)
	freebsd			/@/boot/kernel/kernel
	freebsd_module_elf	/@/boot/kernel/opensolaris.ko
	freebsd_module_elf	/@/boot/kernel/zfs.ko
	freebsd_module		/@/boot/zfs/zpool.cache type=/boot/zfs/zpool.cache
	freebsd_loadenv		/@/boot/device.hints
	set FreeBSD.zfs_load="YES"
	set FreeBSD.vfs.root.mountfrom=zfs:freebsd
	set FreeBSD.vfs.root.mountfrom.options=rw
	set FreeBSD.kern.bootfile=/boot/kernel/kernel
}

And now you have a Mac that quad boots Mac, Linux, Windows, and now FreeBSD. Pretty cool, eh?

One thing that’s important to note is that the partition editor and slice editor used by FreeBSD’s “sysinstall” program, which is often used to install FreeBSD, will destroy your ability to boot Mac OS X, so you need to avoid using it. Manually installing FreeBSD is pretty easy to do, so don’t fret too much. Just use this guide. In my case, I skipped steps 1.3 through 1.5 because I had no intention of using the same partition layout (and their booting method is a pain compared to GRUB2). Before using ‘gpart’ to set up partitions, I first set up my Mac OS X partition using the Mac OS X install disc. Doing so is fairly important, because Mac OS X will behave pretty strangely (or not boot at all) if the EFI System Partition isn’t created in exactly the way Disk Utility creates it. I also deviated from the guide by naming my zpool ‘freebsd’ instead of ‘zroot’.

Note that this guide doesn’t tell you exactly how to do the whole installation, but this is largely because the process is self explanatory. However, it is important to note that in my case, I installed the operating systems in this order: Mac OS X, Linux, FreeBSD, Windows. It’s mandatory to do Mac OS X first, mainly for setting up the first and second partitions. But after getting Mac OS X installed, the rest of the operating systems theoretically can be installed in any order.

If you have questions, please email me and I’ll try to update the article as needed.