Quad Booting Your Mac

March 2nd, 2010 by Steven Noonan   General

UPDATE (March 5, 2010): I’ve updated the article to reflect that I now keep my FreeBSD /boot on the ZFS partition.

You may remember my last article on a similar topic. At the time, I thought any more than three operating systems couldn’t be done on a Mac. But surprisingly, I’ve managed to install four operating systems with relative ease.

This time, I’ve got Mac OS X (10.6.2), Windows 7, Ubuntu Linux 9.10, and FreeBSD 8.0 installed.

Historically, there have been a couple problems with doing quad booting. Most operating systems (Linux and FreeBSD namely) don’t do well when booted directly from EFI, because this boot method skips Apple’s BIOS emulation. This prevents using graphics cards with hardware acceleration enabled, among other things. Being a game developer, this means that EFI-booting Linux and FreeBSD is not an option for me. The second problem is that the MBR/GPT hybrid only allows the MBR side to have at most 4 primary partitions and no extended/logical partitions. In the past, this has meant that things booted under BIOS emulation can only “see” the first 4 partitions unless they have built-in support for GPT.

In Linux’s case, it’s simple to get it to “see” past the 4th partition by compiling the kernel with GPT support. Linux’s boot process on a tri-boot or quad-boot mac goes something like this:

  1. rEFIt switches from EFI-boot mode to BIOS emulation and starts GRUB, using the third partition’s boot sector.
  2. GRUB initializes, and since GRUB only has support for the MBR, it can only see the first 4 partitions.
  3. GRUB loads and boots the Linux kernel from the third on-disk partition (which is Linux’s /boot partition).
  4. The Linux kernel initialises, loads its built-in GPT support and can see past the 4th partition.

So Linux has no problem getting past the MBR’s 4-partition limit. Now, how do we deal with FreeBSD? As you can see from the partition table below, FreeBSD’s partitions are past the MBR’s 4-partition limit (illustrated by the gray line).

Partition Table
Index Size GPT Type Comment
1 200MB EFI System (FAT)
2 125GB HFS+ Mac OS X
3 1GB Basic Data Linux /boot
4 125GB Basic Data Windows 7
5 4GB Basic Data FreeBSD swap
6 100GB Basic Data FreeBSD ZFS
7 4GB Linux Swap
8 60GB Basic Data Linux root
9 60GB Basic Data Linux /home

So how do we boot FreeBSD without using a EFI-based boot? We upgrade GRUB to GRUB2. The newer GRUB2 has support for UFS2 (which is a filesystem FreeBSD’s /boot can use) as well as support for GPT. Ubuntu also has a patched GRUB2 which has support for ZFS. It’s certainly possible to compile their spin of GRUB2 on other Linux distributions, but it’s not exactly a cakewalk. I ended up compiling it myself so that I could use Fedora instead of Ubuntu, though.

All that’s needed to get GRUB to boot FreeBSD (in the case of the partition layout above) is to add this to Ubuntu’s /etc/grub.d/40_custom, and then run ‘update-grub’ as root:

menuentry "FreeBSD 8.0" {
	insmod zfs
	set root=(hd0,6)
	freebsd			/@/boot/kernel/kernel
	freebsd_module_elf	/@/boot/kernel/opensolaris.ko
	freebsd_module_elf	/@/boot/kernel/zfs.ko
	freebsd_module		/@/boot/zfs/zpool.cache type=/boot/zfs/zpool.cache
	freebsd_loadenv		/@/boot/device.hints
	set FreeBSD.zfs_load="YES"
	set FreeBSD.vfs.root.mountfrom=zfs:freebsd
	set FreeBSD.vfs.root.mountfrom.options=rw
	set FreeBSD.kern.bootfile=/boot/kernel/kernel
}

And now you have a Mac that quad boots Mac, Linux, Windows, and now FreeBSD. Pretty cool, eh?

One thing that’s important to note is that the partition editor and slice editor used by FreeBSD’s “sysinstall” program, which is often used to install FreeBSD, will destroy your ability to boot Mac OS X, so you need to avoid using it. Manually installing FreeBSD is pretty easy to do, so don’t fret too much. Just use this guide. In my case, I skipped steps 1.3 through 1.5 because I had no intention of using the same partition layout (and their booting method is a pain compared to GRUB2). Before using ‘gpart’ to set up partitions, I first set up my Mac OS X partition using the Mac OS X install disc. Doing so is fairly important, because Mac OS X will behave pretty strangely (or not boot at all) if the EFI System Partition isn’t created in exactly the way Disk Utility creates it. I also deviated from the guide by naming my zpool ‘freebsd’ instead of ‘zroot’.

Note that this guide doesn’t tell you exactly how to do the whole installation, but this is largely because the process is self explanatory. However, it is important to note that in my case, I installed the operating systems in this order: Mac OS X, Linux, FreeBSD, Windows. It’s mandatory to do Mac OS X first, mainly for setting up the first and second partitions. But after getting Mac OS X installed, the rest of the operating systems theoretically can be installed in any order.

If you have questions, please email me and I’ll try to update the article as needed.

Rewriting Onlink

August 15th, 2009 by Steven Noonan   General

Miah and I decided a few weeks back that the code that Onlink is based on is unmaintainable. The code is riddled with some pretty major problems:

  • Poor design decisions
    Every Person is also a Company? Why?!
  • “magic” numbers littered everywhere
    One example: I tried changing the default values for the Date class to be one second past the UNIX epoch (1/1/1970) to allow for use of standard date functions (difftime, mainly). Unfortunately, this didn’t work too well when missions were generated, because other places determined whether the Date had been set or not by comparing the year to the old default (1000). So all the missions actually expired before they could be displayed, since they appeared to be over 50 years old.
  • OpenGL immediate mode
    According to Alastair, immediate mode is deprecated. He’s been harping on me about using immediate mode for graphics code. Generally speaking, he’s right to harp on this. In most cases, immediate mode is much slower than using the logical alternatives: display lists and vertex buffer objects. It’s also a lot harder to code without immediate mode. But we’ll work through this over time.
  • Bizarre problems that can’t be easily or immediately identified
    The current save issues with Windows XP, for instance.
  • Licensing issues
    It’s difficult to get contributions to the project when every developer has to have obtained the Uplink Developer CD [legally]. It just makes sense for this to be an open source project instead.

Long story short, we’re sick of maintaining code that is unwieldy. It’s time for a proper rewrite.

So on July 22, I started working on Codename “Cerberus”. We have decided that keeping the name “Onlink” anymore is inappropriate, since this is no longer just a mod of Uplink. We don’t know what we’ll call it yet, but for the time being, it’s just “Cerberus”.

I basically took the ARC++ code base, ripped out anything ARC-specific, and started coding on it. My first job is to come up with a decent interface. Miah and I asked the community for suggestions, and we saw some pretty interesting ideas that we may integrate.

Miah and I decided that we needed to start off with a few bullet point goals for the project:

  • Fast
    We want to keep slow code out of our project. We don’t want to cut corners and use poor code, even if it saves on development time. We want to focus on the long-run. Maintainable, high-performance code.
  • Customizable
    We’re going to try to make pretty much everything configurable.
  • Intuitive
    Everything should be easy to figure out and learn, especially for people who haven’t even looked at Onlink before. Following in the footsteps of the developers at Valve, we want to make the game teach the player how to play as they play. Hopefully, you shouldn’t need tutorials or manuals to figure out what you’re doing, if we do this properly.

We have more than that, but we don’t want to give away our content concepts that we’re working on.

Also, Miah and I will be discussing open source licenses for Cerberus sometime between August 29th and September 3rd, and we’ll hopefully be able to publish some code at that point.

So within the next couple weeks, we’ll be posting several different screenshots and user interface demos, and hopefully get some comments and suggestions. Subscribe to this blog’s RSS feed to keep tabs on this!

Crazy Backup Solution

July 13th, 2009 by Steven Noonan   General

I am thinking about attempting to install FreeBSD on my Mac again. The last time I tried, though, it wiped my GUID Partition Table (GPT) and my Master Boot Record (MBR). So basically, it trashed the system and I had to reformat again. It wasn’t lovely. So this time, I decided to back up important things. My source code is already backed up, thanks to Git. My preferences, bookmarks, etc are all backed up by .Mac (oh, excuse me. “MobileMe“), so all that’s really left is my Applications folder and a few miscellaneous documents.

Normally, when backing up my Applications folder, the process is fairly straightforward:

  • Run AppFresh to get the latest versions of my apps.
  • Run Xslimmer to strip out localizations and PowerPC parts.
  • Hand-pick the applications I want to back up and copy them to a network share.

Unfortunately, this is prone to error, especially in the third step. It’s possible to click too quickly when multi-selecting, and open dozens of applications at once. I don’t much like that. So I hacked together a fairly simple (but quite nifty) solution to deal with backing up my applications.

Let’s start with the problem. The problem is that there are several applications that I shouldn’t back up, because they’re provided by the Mac OS X installer. iTunes, Front Row, etc. In fact, I can’t think of a single application made by Apple that doesn’t either come with the OS or need an installer in order to operate correctly (i.e. Aperture, Logic, Final Cut, iLife, iWork, etc). So these applications need to be excluded while backing up.

So I started with a simple script that filters through the applications I’ve got and finds ones that need to be backed up:

filter.sh

#!/bin/bash

BANLIST="$(cat filter-blacklist)"
WHITELIST="$(cat filter-whitelist)"

# We don't want spaces to muck up the paths.
export IFS=$'\n'

rm -f backup-queue
for a in $(find /Applications -maxdepth 1 -depth 1 -type d | sed 's/\/Applications\///g' | sort -f); do

	# Simple exclusion rule for Apple-provided apps.
	EXCLUDE=$(cat "$a/Contents/Info.plist" 2> /dev/null | grep -A 1 CFBundleIdentifier | grep com.apple)

	# We also run this through a blacklist and a whitelist, just
	# in case there's something that we _did_ want to be banned
	# or vice versa.
	if [ "$EXCLUDE" == "" ]; then
		BANNED=0
		for b in $BANLIST; do
			if [ "$a" == "$b" ]; then
				BANNED=1
				break
			fi
		done
	else
		BANNED=1
		for b in $WHITELIST; do
			if [ "$a" == "$b" ]; then
				BANNED=0
				break
			fi
		done
	fi

	# JUDGEMENT TIME
	if [ $BANNED -eq 0 ]; then
		echo $a OK
		echo $a >> backup-queue
	else
		echo $a BANNED
	fi
done

The script requires two files. A whitelist (applications that should be backed up, but are excluded by the filter) and a blacklist (applications that shouldn’t be backed up, but pass the filter). Here are mine:

filter-blacklist

Adobe Bridge CS4
Adobe Device Central CS4
Adobe Dreamweaver CS4
Adobe Drive CS4
Adobe Extension Manager CS4
Adobe Flash CS4
Adobe Illustrator CS4
Adobe Media Encoder CS4
Adobe Media Player.app
Adobe Photoshop CS4
AppleScript
Microsoft Office 2008
NetBeans.app
Utilities
VirtualBox.app
VMware Fusion.app

 

filter-whitelist

Plasma Pong.app

A bit of explanation on my blacklist and whitelist… Plasma Pong.app is on the whitelist because the author specified that the CFBundleIdentifier is ‘com.apple.plasmapong’, which gets caught by the anti-Apple app filter (a CFBundleIdentifier of ‘com.plasmapong.plasmapong’ would be better). The folders/applications listed on the blacklist are ones which are either provided with the OS (‘AppleScript’, ‘Utilities’), or require an installer to function (‘NetBeans’, ‘VirtualBox’, ‘VMWare Fusion’).

So anyway, the script uses these two files and its filter to figure out what apps should be included or excluded. Let’s look at what the script outputs in my case:

Alcarin:Applications steven$ ./filter.sh
0xED.app OK
Address Book.app BANNED
Adium.app OK
Adobe Bridge CS4 BANNED
Adobe Device Central CS4 BANNED
Adobe Dreamweaver CS4 BANNED
Adobe Extension Manager CS4 BANNED
Adobe Flash CS4 BANNED
Adobe Illustrator CS4 BANNED
Adobe Media Encoder CS4 BANNED
Adobe Media Player.app BANNED
Adobe Photoshop CS4 BANNED
Angband.app OK
Anxiety.app OK
Aperture.app BANNED
AppFresh.app OK
AppleScript BANNED
AppZapper.app OK
Arora.app OK
Audacity.app OK
Automator.app BANNED
BetterZip.app OK
blender 2.48a OK
Braid.app OK
Calculator.app BANNED
Canary.app OK
CandyBar.app OK
Chess.app BANNED
Chmox.app OK
coconutBattery.app OK
coconutIdentityCard.app OK
Coda OK
Colloquy.app OK
CrossOver Games.app OK
CrossOver.app OK
Cyberduck.app OK
DAA Converter.app OK
Darwinia.app OK
Dashboard.app BANNED
Dictionary.app BANNED
Disco.app OK
Dock Library.app OK
DOSBox.app OK
Doukutsu.app OK
DropCopy.app OK
DVD Player.app BANNED
Dwarf Fortress 0.28.181.40d OK
Expose.app BANNED
Firefox.app OK
Flip4Mac OK
Flock.app OK
Font Book.app BANNED
Freeciv.app OK
Front Row.app BANNED
FrostWire.app OK
Geekbench (64-bit).app OK
Geekbench (Rosetta).app OK
Geekbench.app OK
Gimp.app OK
GrandPerspective.app OK
GridWarsOSX OK
Hacker Evolution Untold.app OK
HandBrake.app OK
iCal.app BANNED
iChat.app BANNED
Image Capture.app BANNED
Inkscape.app OK
iPodDisk.app OK
iStumbler.app OK
iSync.app BANNED
iTunes.app BANNED
Jaikoz.app OK
Leopard Cache Cleaner.app OK
LiquidMac.app OK
Little Snitch Configuration.app OK
MacHeist Chat.app OK
MacPorts OK
Mactracker.app OK
Mail.app BANNED
Microsoft Office 2008 BANNED
Multiwinia.app OK
NetBeans OK
NetNewsWire.app OK
Nocturne.app OK
Opera 10 beta.app OK
Opera.app OK
Photo Booth.app BANNED
Picasa.app OK
Picturesque.app OK
Pixelmator.app OK
Plasma Pong.app OK
Preview.app BANNED
Privateer - Ascii Sector OK
Quicksilver.app OK
QuickTime Player.app BANNED
RealPlayer.app OK
Safari.app BANNED
Scribus.app OK
SeaMonkey.app OK
Senuti.app OK
Sequel Pro.app OK
Shimo.app OK
SiteSucker.app OK
SketchFighter 4000 Alpha OK
Skitch.app OK
Skype.app OK
smcFanControl.app OK
Smultron.app OK
Spaces.app BANNED
Speed Download 5 OK
Stickies.app BANNED
SubEthaEdit.app OK
System Preferences.app BANNED
TextEdit.app BANNED
The Unarchiver.app OK
Time Machine.app BANNED
Transmission.app OK
TrueCrypt.app OK
Twitterrific.app OK
Unangband.app OK
Utilities BANNED
Ventrilo.app OK
VLC.app OK
VMware Fusion.app BANNED
Vuze.app OK
WebKit.app OK
World of Goo.app OK
X-Chat Aqua.app OK
Xbench.app OK
Xslimmer.app OK
Yep.app OK
ZAngband.app OK
Alcarin:Applications steven$

Alright, looks good. filter.sh also generates a file called ‘backup-queue’ which contains all the apps which passed the filter. So now, step two is a two-part process:

  • Archive the apps (gzipped tarball works for this)
  • Copy the tarballs to a network share

So I created a couple scripts for this (they can be run in parallel, which is fantastic for a couple reasons which I’ll outline below).

backup.sh

#!/bin/bash

cd /Applications

QUEUE="$(cat backup-queue 2> /dev/null)"

# We don't want spaces to muck up the paths.
export IFS=$'\n'

echo "Running backup queue..."

rm -iv *.tar.gz.lock *.tar.gz

for a in $QUEUE; do

	echo -n "Creating $a.tar.gz... "

	# Create a lock so that the move script won't touch
	# the file until we're done with it here.
	touch "$a.tar.gz.lock"

	# Do the actual work.
	tar -czpf "$a.tar.gz" "$a"

	# Mark this tarball complete.
	rm -f "$a.tar.gz.lock"

	echo "OK"

done

echo "All done."

 

move.sh

#!/bin/bash

DESTINATION="$1"

# The user doesn't know what he's doing, clearly.
if [ "$DESTINATION" == "" ]; then
	echo "Please provide a destination directory."
	exit 1
fi

# I spoke too soon. _NOW_ the user has no idea what they're doing.
if [[ ( ! -d "$DESTINATION" ) || ( ! -w "$DESTINATION" ) ]]; then
	echo "Destination specified is not a writable directory."
	exit 1
fi

# We don't want spaces to muck up the paths.
export IFS=$'\n'

cd /Applications

while true; do

	# Stays zero unless something actually gets moved.
	DIDWORK=0

	QUEUE="$(ls | grep tar.gz$)"
	for a in $QUEUE; do

		# If the lock file for the tarball doesn't exist,
		# we assume that backup.sh has finished its work.
		if [ ! -f "$a.lock" ]; then
			DIDWORK=1
			echo "Moving $a to $DESTINATION/..."

			# Finally move it to the backup storage
			# directory.
			mv "$a" $DESTINATION/ &> /dev/null
		fi
	done

	# We sleep if no work is done because otherwise
	# this infinite loop would waste quite a few CPU
	# cycles.
	if [ $DIDWORK -eq 0 ]; then
		echo "Nothing to do. Sleeping..."
		sleep 2.5s
	fi

done

You might wonder why I didn’t just have it tarball directly to the remote server. The reason is fairly simple. If the network latency is bad enough, it won’t be able to transfer over the network fast enough and the archiving process grinds to a halt while waiting for the network to catch up. And if the bottleneck happens to be in your archiving speed, this doesn’t adversely affect it, either. This parallel method ensures that the maximum amount of work is being done at a time.

8-bit ANTICs, Part One

July 5th, 2009 by Miah   General

When I was offered an Atari 2600 Junior for perhaps the most awesome reason imaginable, I could hardly believe my luck. I wasn’t able to have one when they were new, and the only time I got to play one was while I was at a grandmother’s house. I still have my NES, but it’s a different sort of thing. One plays an Atari for the experience. Or the nostalgia. As for me, I haven’t played one since I was about four, so it’s a little from column A and a little from column B.

So when a package came for me last week, I was thrilled, knowing what it was. Opening the package was a little less awesome, however. The system was a damaged one, and it was almost certain that all the visible damage was done in transit. Pieces were rattling inside the system and in the box.

100_0609It would suffice to say that the system isn’t exactly working at this moment, but I’ve not lost hope for it. The power light itself did light up for a brief moment before dying out, so I think that this system is savable. It’s sort of out of my scope right now, however, because I don’t know what’s wrong with it. That didn’t stop me from opening it up to take a look though.

Now, I know that things in the mail aren’t typically handled with kid gloves, but there’s one thing I’m particularly curious about, and that is what the hell was done to this package. The system was packed decently, but not superfluously. Yet the corner was, for lack of a better term, shattered. This could be due to the age of the plastic, but could be the sheer force that the package hit.

Case in point: How hard would you as the reader expect the system to be hit if I said peg A and switch B were physically separated?

100_0614Well, they were. I had though the switch was busted, but as it would happen, the peg was set to the “color” mode and the switch was jammed in the “B/W” mode. It’s just happy that the switch itself wasn’t broken.

It doesn’t make me at all happy, though. This isn’t the first time that the post office has manhandled my packages. The prior time was a full desktop computer mailed from southern Texas and showed up with a fractured motherboard, bent GPU, and shattered case. Best part was that this particular one was an insured package, and the post office refused entirely to honour that insurance.

But I digress.

At this point, I am simply wondering what might be broken. Seeing as I have never seen the insides to one of these things while functional, I really don’t know what might have been knocked loose or is now missing. I really am not thrilled with this turnout so far, but am hoping there’s something to be had from it.

I am welcoming any suggestion for repair. I have a few thoughts. In the first image below, it would appear that one piece here that I assume acts as a ground is separated. There was no screw to anchor it down, however, so that might have been incidental. The third and fourth pictures are the obverse and reverse of the main circuit board. Hopefully someone will have some insight.

A person can hope, right?

100_0610100_0611100_0612100_0613

(with apologies and givings of thanks nevertheless to frigginjoe)

The Future Definition of “Cross Platform”

June 25th, 2009 by Miah   General

Preamble

I would like to spend some time with the community to discuss a new technology. Well, not really “new” per se (it’s been in use since 2007), but it’s certainly something that’s recently gone under the crosshairs of my ire.

I speak of the Cider system, a method to deploy Windows games to the Mac cheaply and quickly.

So let’s have a discussion on this system. As usual, I hope you like text.

What is Cider?

From the TransGaming site: “TransGaming’s Ciderâ„¢ Portability Engine is a proprietary technology that allows PC games to be enabled on Apple’s Intel Macs without the traditionally expensive and arduous need to redevelop a game from the ground-up. Cider acts as a “wrapper” around the PC game dynamically translating PC API calls to the Mac OS X operating system. As such, games can be enabled with Cider in a matter of days to weeks as opposed to the typical man years that traditional development takes.

Why use Cider? (or: what’s the point?)

Cider is often used as a too-little-too-late correction for a major mistake made in early development: the use of Windows-only APIs like .NET, Direct3D, DirectSound, GDI, etc. Programs written to use these APIs can’t run on platforms other than Windows without the use of an emulation layer.

The .NET Framework – Strictly speaking, i don’t find a lot of utility in the .NET framework and tend to use other tools that the framework was meant to simplify. However, it is a best-option in a lot of circumstances for game developers. Portability is probably the biggest thing to this. By “portability”, we speak strictly of Microsoft-only systems; desktop and handheld devices, but the big deal here for the gaming industry is the Xbox portability. Since it’s a gaming platform, it makes it possible to have one’s product be both a PC game and a console game without much additional overhead. Compare this to the Wii method which includes a separate system the size of a 4U server.

DirectX – DirectX is a multitude of utilities from complex graphic rendering to control handling (keyboard, mice, and game control pads) to sound and music playback. Along with the .NET framework, DirectX is used because it works on Window 95 and above, Windows CE (to a limited extent), and most importantly, the Xbox.

Who uses Cider? (or: Dear God, why?)

At this point, game companies, rather exclusively, make use of Cider. You might know a few games: The Sims 3, Spore, City of Heroes and EVE Online to name a few.

Cider’s aim is to make porting a Windows game to Mac a lot easier, eliminating many hours of coding time. For instance, DirectX isn’t supported on Mac OS X, so the graphics API would need to be remapped to use OpenGL. Generally speaking, Mac games from major distributors don’t use Cocoa or Quartz, but rather OpenGL. The reason for this is simple: OpenGL works on pretty much anything. Mac OS X, Linux, FreeBSD, Windows, the iPhone, etc.

By why use DirectX at all if compatibility is an issue? There’s a few reasons. Several years ago, DirectX was a lot better in terms of performance than OpenGL. As a result, several companies spent good sums of money to train their developers to use DirectX Instead of switching to OpenGL, companies generally avoid the extra cost of training and continue to use DirectX. Once a game is done, it’s passed along to a separate team that does the port. Because all the code is in place as far as what a function call can do, it’s basically a codeswap; DirectX calls translated to OpenGL calls. Of course, it’s a lot more complex than this, but this is the general idea to the average layman; consider it like a game with a story written in English being translated to Japanese.

The process really is a long one, however, so a lot of money and hours need to go into it.

So what’s the problem? (or: The Domino Effect)

Know the saying “you get what you pay for”? It doesn’t apply here. Mac gamers pay as much as Windows players, and find that their experience is really diminished. That’s because Cider runs as an emulation layer.

There are various types of emulators. Full hardware emulators (i.e. Bochs, QEMU, etc), virtualizers (i.e. VMware, VirtualBox), and API wrappers (i.e. Wine, Crossover, Cider, etc). Full hardware emulators deal with translating machine code between two completely different architectures. This is perhaps the most complex and painful part of emulation. It causes tremendous overhead unless it’s done very well (i.e. a combination of dynamic recompilation and jitting). In the case of an API wrapper, the overhead shouldn’t be as much. The hard work is already done. The code is already native (x86 or x86_64).

Unfortunately, nobody has yet done an API wrapper particularly well. Current API wrappers such as Wine run much slower than they should. A majority of the Windows APIs can be mapped to POSIX API calls without much effort. Windows threads become pthreads. Windows critical sections become pthread mutexes. DirectX calls become OpenGL calls. But the overhead of an API wrapper such as Cider is much more tremendous than one might expect.

Is there much of a performance impact? Oh yes. Cider doesn’t care if you have more than one processor. Your state-of-the-art graphics card will be chugging along as if it were a 3-year old model. OpenCL, which is coming out in 10.6? Worthless. The Physical Address Extension (PAE) that allows you to run on more than 4GB of RAM? Likely going to be ignored. That’s to name just the hardware limitations.

On top of this, there are software issues. The method of coding to make DirectX fast isn’t the same method as to make OpenGL fast. It’s going to run slower because the original code for the game is totally unmodified. Additionally, and probably the biggest hit to performance, is the translation of the common WinAPI functions, such as page fault handling. Page fault handling is a very basic need; it’s not unusual to see a page fault delta of 20,000 per second on a very active application (i.e. games, which is what this is all about!).

Transgaming, the company that makes Cider and Cedega, have based their products on Wine for Linux. Since Transgaming is making these products with gaming use in mind, we might expect to see that Cedega outperforms Wine. Let’s run some basic micro-benchmarks and see how Cedega and Wine perform compared to native execution on Windows XP. Tests with the largest differences are shown in bold (thanks to Darek Mihocka for the great micro-benchmarking tool!).

Windows XP on a 2.33GHz Intel Core 2 Duo:

test 1 os crit 2 : 237 ms, 79000 ps/instr, 12 MIPS, 184.3 clk
test 2 os try ok : 11 ms, 3666 ps/instr, 272 MIPS, 8.5 clk
test 3 os pg flt : 71 ms, 2366666 ps/instr, 0.422 MIPS, 5528.4 clk
test 4 os virtqr : 14 ms, 466666 ps/instr, 2 MIPS, 1089.1 clk
test 5 os virtpr : 16 ms, 533333 ps/instr, 1 MIPS, 1244.2 clk
test 6 os isbadr : 150 ms, 5000000 ps/instr, 0.200 MIPS, 11665.0 clk
test 7 os isgood : 47 ms, 15666 ps/instr, 63 MIPS, 36.5 clk
test 8 os vxpflt : 71 ms, 2366666 ps/instr, 0.422 MIPS, 5528.4 clk
test 9 CmpExch 2 : 214 ms, 71333 ps/instr, 14 MIPS, 166.4 clk
test 10 GetInpSt : 9 ms, 3000 ps/instr, 333 MIPS, 6.9 clk
test 11 GetQStat : 1186 ms, 395333 ps/instr, 2 MIPS, 922.4 clk
test 12 PeekMsg : 1377 ms, 459000 ps/instr, 2 MIPS, 1071.1 clk
test 13 MemMap 2 : 107 ms, 3566666 ps/instr, 0.280 MIPS, 8332.1 clk
test 14 QueryPrf : 5243 ms, 1747666 ps/instr, 0.572 MIPS, 4078.6 clk

Linux running Wine v1.0.1, on the same 2.33 GHz Intel Core 2 Duo:

test 1 os crit 2 : 295 ms, 98333 ps/instr, 10 MIPS, 229.4 clk (1.24x slower)
test 2 os try ok : 12 ms, 4000 ps/instr, 250 MIPS, 9.3 clk (1.09x slower)
test 3 os pg flt : 163 ms, 5433333 ps/instr, 0.184 MIPS, 12679.3 clk (2.29x slower)
test 4 os virtqr : 44 ms, 1466666 ps/instr, 0.681 MIPS, 3425.8 clk (3.14x slower)
test 5 os virtpr : 91 ms, 3033333 ps/instr, 0.329 MIPS, 7091.1 clk (5.68x slower)
test 6 os isbadr : 156 ms, 5200000 ps/instr, 0.192 MIPS, 12151.0 clk (1.04x slower)
test 7 os isgood : 71 ms, 23666 ps/instr, 42 MIPS, 55.2 clk (1.51x slower)
test 8 os vxpflt : 173 ms, 5766666 ps/instr, 0.173 MIPS, 13485.5 clk (2.43x slower)
test 9 CmpExch 2 : 217 ms, 72333 ps/instr, 13 MIPS, 168.7 clk (2.43x slower)
test 10 GetInpSt : 34990 ms, 11663333 ps/instr, 0.085 MIPS, 27447.0 clk (3887.78x slower)
test 11 GetQStat : 34967 ms, 11655666 ps/instr, 0.085 MIPS, 27447.0 clk (29.48x slower)
test 12 PeekMsg : 42974 ms, 14324666 ps/instr, 0.069 MIPS, 33811.5 clk (31.20x slower)
test 13 MemMap 2 : 2545 ms, 84833333 ps/instr, 0.011 MIPS, 212090.9 clk (23.78x slower)
test 14 QueryPrf : 4723 ms, 1574333 ps/instr, 0.635 MIPS, 3674.0 clk (1.12x faster)

Linux running Cedega 7.3.1 on the same 2.33GHz Core 2 Duo:

test 1 os crit 2 : 286 ms, 95333 ps/instr, 10 MIPS, 222.4 clk (1.21x slower)
test 2 os try ok : 24 ms, 8000 ps/instr, 125 MIPS, 18.6 clk (2.18x slower)
test 3 os pg flt : 1995 ms, 66500000 ps/instr, 0.015 MIPS, 155533.3 clk (28.10x slower)
test 4 os virtqr : 7 ms, 233333 ps/instr, 4 MIPS, 544.4 clk (2.0x faster)
test 5 os virtpr : 96 ms, 3200000 ps/instr, 0.312 MIPS, 7477.5 clk (6.00x slower)
test 6 os isbadr : 1975 ms, 65833333 ps/instr, 0.015 MIPS, 155533.3 clk (13.16x slower)
test 7 os isgood : 102 ms, 34000 ps/instr, 29 MIPS, 79.3 clk (6.00x slower)
test 8 os vxpflt: FAILED!! (games using vectored exception handling would crash)
test 9 CmpExch 2 : 289 ms, 96333 ps/instr, 10 MIPS, 224.7 clk (1.35x slower)
test 10 GetInpSt : 4925 ms, 1641666 ps/instr, 0.609 MIPS, 3830.8 clk (547.22x slower)
test 11 GetQStat : 4796 ms, 1598666 ps/instr, 0.625 MIPS, 3732.8 clk (4.04x slower)
test 12 PeekMsg : 41978 ms, 13992666 ps/instr, 0.071 MIPS, 32859.1 clk (30.49x slower)
test 13 MemMap 2 : 2983 ms, 99433333 ps/instr, 0.010 MIPS, 233300.0 clk (27.88x slower)
test 14 QueryPrf : 4078 ms, 1359333 ps/instr, 0.735 MIPS, 3174.1 clk (1.09x faster)

Oddly enough, the use of Cider is actually worse than using the open-source Wine. The result: a far less enjoyable experience, but probably more enjoyable than, say, running a Windows game in Parallels or VMWare Fusion.

In short, you get what the game developer paid for. Low performance for a low price.

If you’ve play Spore on the Mac platform at all (which I have and I’m throughly unimpressed with the game), then you’re probably thinking that I’m taking things out of proportion. Well no, I’m really not. Games like Spore and the Sims 3 aren’t exactly what we call graphics intensive, but games like EVE Online have the potential for very intensive sequences (namely battles). Here’s the bottom line: you’ll never find games like Bioshock, Crysis or even Portal running on the Mac (more on Portal later).

Should I be concerned? (or: So what’s the problem exactly?)

The use of Cider drops the bottom line. Little additional development is required and the price tag is a lot less than porting to native code.

Should you be concerned? Oh yes. Cider encourages developers to be lazy in the porting process, allowing them to exploit the Mac and Linux gaming markets with underachieving products.

In general, Macs can be good gaming platforms, specifically due to the hardware used in Macs on a given day. Not the best hardware, but certainly among the top. If you’re going to run a Windows game in BootCamp, then you’re going to be pretty well off on high detail modes, even on the laptop models (at least the newer ones). Using a Cider port, however, means that you might be getting intermittent frames, even on low detail modes.

This isn’t really a big deal though if you don’t play games, right? I beg to differ. Games are fairly complex in the emulation layer support they require. The main reason that we’re seeing this behaviour in games is because it’s the one industry that lacks porting the most.

It’s fairly safe to say that constructing a wrapper for other applications would be a lot easier, particularly if the wrapper is not a lot more than Wine. Imagine if other corporations started adopting the Cider method of porting. What could we see?

Imagine Adobe Photoshop or Premiere ported with a Cider wrapper. Or Autodesk Maya, Blender or Bryce. All of these have ports in native code, but what if, in the recessed economy, these companies decide to go the Cider route to cut the bottom line?

This would be a very good way to effectively ruin the Mac platform. While more apps could theoretically be used on Mac OS X, they would all be running in an emulation layer, cutting back on performance. If you use these utilities for a job, there’s a good chance that you would think twice before buying a Mac due to the performance issues.

Apple is trying to make a habit out of optimization. It’s the key focus of Snow Leopard. Unfortunately, no amount of optimized code on the part of Apple will matter if companies start favouring the Cider method outside the gaming industry. Already, gamers that were originally excited to hear of big names like EA coming to the Mac platform are instead feeling disenfranchised rather than empowered. Now imagine if this was happening in the video industry or the graphical arts industry.

It’s been said by some others that Cider could means the death of Mac gaming as we know it, but the potential here is so much more dangerous. It also has lasting implications.

Unintended consequences (or: Piracy Made Easy)

I stated before that I was going to get back to Portal, and now I am. There has been a Cider port of Portal running around for some time now, but it’s not based from Valve.

It is in fact a product of a new fresh mode of piracy. The advent of Cider has actually done more for the piracy movement than anyone else. Generally speaking, pirates understand that they’re already getting more than they are paying for, so certain graphical issues (some things won’t render, namely shaders) aren’t that big of a deal.

However, Portal and Half Life 2 are both critically acclaimed and are on many lists for people as a “must play at least once”. It would thus make sense that they would be targets for piracy. With Cider, that which wasn’t possible without a name brand virtualizer is now possible to run on the Mac, albeit without the quality that you would expect from a natural port.

Pirates don’t care. Free is free and it’s hard to complain when a cracked product is perfectly playable, even if it’s not pristine in graphical quality.

Afterword (or: Digression)

Cider really has a lot going for it in the future, and that’s inescapable. Being able to make any graphically dependent program truly cross platform is really no small feat, and the near-guarantee that a product can work on multiple platforms without significant code changes is a rather tempting reason to use it.

This however really brings to bear the use of the term “cross compatible”. A program that requires a virtualizer certainly does fall under the definition, but something like Cider isn’t so easy to classify. It’s not really even a grey area; it’s more like a splotchy murk. I worry that there might be a point in time that it start affecting the work I do. I rather enjoy the Mac platform, but I’d be damned if I use it to run a dozen wrapped applications instead of a dozen applications with native code.

I mean, the Mac version of Word is bad enough as it is.

Stylish Yet Functional

May 29th, 2009 by Steven Noonan   General

I decided my desktop’s current appearance was worth sharing. Enjoy.

My MacBook Pro's desktop

The Mystery of the Horrible Boot Speed

May 21st, 2009 by Steven Noonan   General

My Mom recently complained to me that her Hewlett-Packard brand PC was booting awfully slowly. She blamed it on my actions, claiming that it had to be the copy of Safari that I installed. She’s horrified to install anything on her computer for fear that it will “screw it up” or “slow it down”. In her case, it’s not a terribly unreasonable fear. If something does go wrong, she doesn’t know what to do to fix it. But blaming Safari for this problem did not make sense in the slightest, so I decided to take a peek under the hood and see why Windows Vista would boot so slowly on such a decent computer (besides the obvious, of course).

It took a while to find the Windows Vista equivalent to Windows XP’s BootVis tool, but apparently they made a new set of utilities for the task, called the Microsoft Windows Performance Toolkit. I won’t go into the usage of it, because you can Google it if you need that information. But I will show you how to interpret the results you get from it, and expose the culprit of this particular performance nightmare.

The first run showed a pretty horrendous sight:

Terrible Boot Graph

Unfortunately the first results were rather worthless, as they were absolutely littered with disk input/output caused by AVG:

Stupid AVG.

So I uninstalled AVG and ran the test a second time. This time, I got some conclusive results:

Okay, now we're talkin'

There are a few things to note about this graph. First of all, the core system boot sequence is shown to be taking 75 seconds (by “core system” I mean the time it takes between BIOS and loading a login screen). This is horrifying. Consider for a moment that Linux can finish the system boot in less than 20 seconds and reach a fully logged in graphical user interface in an additional 5-15 seconds. So in perspective, a total boot time of roughly 190 seconds (75 system + 115 userspace) is absolutely terrible.

It’s time to figure out the proximate cause. I noticed in the graph above that there is roughly a 20 second period during the system boot where there is minimal CPU usage, very few disk I/O requests, but the disk utilization is maxed out. This tells me that it’s doing a lot of slow requests which stall the system. I decided to look at the disk access graph, which shows where the drive is physically reading/storing data at a point in time.

Good lord...Good lord...

Criminy. That 20 second stall (highlighted in the second image above) is being caused by thrashing between some files that are physically very far apart on disk. Here’s a zoom on that section:

Good lord...

Well, details are necessary now. We need to know what files it is thrashing between (and why) in order to be able to do anything about it.

Registry disk-thrash?

Aha! Success! Er, well, maybe not. The files are the “COMPONENTS” and “SOFTWARE” registry hives. This is pretty much bad news because the Windows defragmenter can’t resolve this problem for two reasons:

  • Microsoft’s defragmenter doesn’t optimize for the most part. It’s largely intended for quick-and-dirty defragmentation.
  • Even if it did optimize, it cannot move files that are open for reading or writing.

It’s very strange though, that Windows didn’t simply read the two files in their entirety in two large reads instead of this silly thousands-of-tiny-reads crap. I think this could very well be a bug in Windows Vista. Come on, Microsoft. What were you thinking? A programming 101 student could tell you that a large disk read is faster than lots of smaller ones.

So in order to work around the problem, I first needed to be able to defragment my Mom’s computer’s hard disk completely without running the installed operating system. It’s unfortunate she doesn’t have a Mac, because if she did, we could simply use Target Disk Mode which basically makes the computer behave like an external FireWire drive; all that would be necessary in such a case is hooking it up to another machine as you would with an external hard drive, and then defragment that way. Unfortunately for her, the easiest way to do this was to repartition the drive so that we could install a temporary copy of Windows in a second partition.

The second issue is that we needed to be able to optimize the drive, and not just defragment it. By “optimize”, I mean physically arranging files on disk in such a way that they’re quickly accessed during the system’s boot sequence. This is different from defragmentation, which simply consolidates split files. Thankfully, there’s an awesome open source defragmenter available called JkDefrag, which is capable of optimizing as well as defragmenting. Sweet!

So I installed Windows 7 RC1 onto another partition (created via Disk Management), and then ran JkDefrag with the command-line option ‘-a 7′, which tells JkDefrag to optimize by arranging the files alphabetically based on their path and file name. This should cluster the contents of folders together, making accesses much faster.

So after a couple hours, it finally became bored and finished defragmenting, so I rebooted to re-time Windows Vista’s boot sequence. It was noticeably faster already, but I needed numbers. And I was quite pleased with the results:

A beautiful sight

Note that there’s no obvious point at which a bottleneck is noticeable. Neither the CPU nor the hard drive are maxed out for too long at any one point. And then there’s the physical layout access graph:

More beauty.

Awesome. JkDefrag did its job and the files are much better clustered. Better yet, the raw numbers are even more impressive. The system boot has been reduced to 45 seconds, which is 60% of the time it took previously. Based on the disk and CPU activity, the machine’s boot is complete after about 90 seconds, which is 47% of the time it took previously. (Keep in mind that xbootmgr continues measuring for 120 seconds after the user logs in, so we can’t just use the last time marker on the boot graph as the time it takes to boot. You have to look at disk activity and CPU utilization, and figuring out the point where the constant activity ceases and the machine begins to settle.)

So now my mom’s machine boots pretty quickly. Now I need to move on to my next project: Operation: Get Mom to Stop Using Internet Explorer.

Firefox 3 OpenSSL Woes

May 15th, 2009 by Steven Noonan   General

Well, that was frustrating. I stayed up late last night trying to figure out why my Apache 2.2 server was misbehaving, and it ended up being a Firefox 3 bug. Or more specifically, a bug in the old OpenSSL library that Firefox 3 is statically linked to.

I tried to view our cgit page, and only got this:

And if I refreshed a few times, I got this ’ssl_error_rx_unexpected_change_cipher’ error:

But strangely, if I used Safari, I got exactly what I should have:

I finally figured out that Firefox was screwing up when using the TLS 1.0 protocol. So all I had to do was edit my Apache 2.2 httpd.conf to have this line:

SSLProtocol -SSLv2 +SSLv3 -TLSv1

I don’t like this solution though. The Mozilla Firefox team should release a build of Firefox that is statically linked to a newer version of the OpenSSL library which doesn’t suffer from this bug. I much prefer TLS because of the added security it provides, so I ended up with a line that opted for security over broken SSL library compatibility:

SSLProtocol -SSLv2 -SSLv3 +TLSv1

Also note that I disable SSL v2 as well bacause it’s widely known to be inherently flawed.

Fallout 3 Needed More Work

December 13th, 2008 by Steven Noonan   Gaming

Well, I just finished Fallout 3. It’s an interesting game, and I greatly enjoyed a large portion of it. I especially appreciate how Bethesda built Fallout 3 on the same game engine that Oblivion is based on. Some things, however, really bug me.

One, a majority of the game dialogue is bland and unexciting. This is particularly bad for the stat-checked dialogue. For instance, there’s one marked “[Intelligence]“, which is supposed to be a point in which you impart some keen insight. Unfortunately, the “insight” is actually quite weak, and tends to merely restate the obvious. A tag like “[Captain Obvious]” would be far more true to the actual insight given. Another problem with the dialogue is that non-player characters are so easily swayed. A non-player character could be steadfastly against something, but as long as your Speech skill is high enough, you can convince them to jump off a bridge. Dialogue that involves convincing an NPC about something goes something like this:

Player: You should do this.
NPC: No! It goes against the way I have been brought up, and against everything I believe!
Player: [Speech 100%] You really should do this.
NPC: OK.

I wish this was a joke, but it really is this bad.

Two, the open-endedness seemed far too limited. In TES IV: Oblivion, you could finish the main quest and still have many, many hours of enjoyable gameplay left, largely because of the sheer number of quests. Not so in Fallout 3. You finish the main quest and it’s game over, regardless of whether or not your character does the noble self-sacrifice thing.

I would have liked a better conclusion to the Trouble on the Homefront quest, which really is not very well written. In the way I ended the quest, the player character’s lifelong friend Amata becomes Overseer of Vault 101. Unfortunately, her character does an about-face and basically takes on the personality of her father (the previous Overseer), and exiles the player from Vault 101 forever. I would have liked to see a better conclusion to this quest. Amata’s stated goal, before her character’s about-face, was to open the vault for trade and so forth. With the way the quest ends, she closes the vault for the remainder of the game. It would have made much more sense to me that Vault 101 would be closed for an in-game week or two, to allow her to restore order and bring the vault back to its former state, and eventually open, allowing the player to revisit the vault, old friends, etc. It would also have been more interesting to see residents of Vault 101 visiting Megaton or other nearby places for supplies. Other people have noted their disappointment for their results for the quest as well.

Despite these negative points, I found Fallout 3 to be generally enjoyable. The graphics were superb and the landscape was exactly what I would expect from a post nuclear holocaust environment. The sounds were nicely done as well, and it’s nice to see that Bethesda got more than a handful of voice actors for Fallout 3.

I think Fallout 3 had a lot more potential, and I’m hoping that the downloadable content packs will add something substantial to the game, unlike, for instance, this.

Microsoft Service Packs

November 19th, 2008 by Steven Noonan   General

I just installed service pack 1 on a 64-bit Windows Vista machine, and it stupidly took forever and a half to install. It also took multiple reboot cycles, something that wasn’t needed in previous Windows NT operating systems (2000 and XP included). I think Microsoft’s going the wrong direction with Windows service packs, because it is now ridiculously hard to integrate into a customized Windows disc. With Windows XP service packs, it was simple. You’d copy the files to the hard drive, slipstream, and burn the disc. The whole process took about 20 minutes. With Vista, it’s an interminable procedure requiring third party software (vLite). The process takes an hour or more.

Another service pack that I’ve had annoying troubles with is Visual Studio 2008 SP1. It takes more than an hour to install. In fact, it’s still installing on this machine. I wouldn’t be surprised if it’s actually stuck, because the progress bar hasn’t moved in 20 minutes or so.

Why the hell do Visual Studio service packs take forever to install? I assume the problem with 2008 SP1 is similar to the 2005 SP1 issue, which required a very ugly hack to get around it. I mean, it works, but it just takes forever. With Visual Studio 6, you could install a service pack in less than 5 minutes. This process has ballooned to over an hour. Why?!

Another thing that’s bugging me is that Visual Studio 2008 feels slower than molasses. Apparently I’m not alone in noticing this.

Microsoft’s stuff is just seeming less and less appealing. I can get a decent development machine running Mac OS X set up in about an hour or maybe two (including Xcode, Git, development libraries [SDL, etc], CMake, Smultron, etc). On Windows, It’s taken me at least 8 hours so far. I’ve been installing Windows, Visual Studio, service packs and other updates constantly. And I mean constantly. I haven’t let the machine idle once.

Microsoft! Give up on all this stupidity you’ve been adding to Windows, Visual Studio, and Office for Mac! Slim your applications. Make performance and size a priority. To ignore performance will be an absolutely fatal mistake for you. The only teams I think really have it together right now are the Microsoft Office (Windows) team, and the Xbox team. Their stuff installs easily, updates easily, and runs smoothly. Why can’t the rest of Microsoft get this crap right?!