Anomalous Anomaly

Preamble

I would like to spend some time with the community to discuss a new technology. Well, not really “new” per se (it’s been in use since 2007), but it’s certainly something that’s recently gone under the crosshairs of my ire.

I speak of the Cider system, a method to deploy Windows games to the Mac cheaply and quickly.

So let’s have a discussion on this system. As usual, I hope you like text.

What is Cider?

From the TransGaming site: “TransGaming’s Cider™ Portability Engine is a proprietary technology that allows PC games to be enabled on Apple’s Intel Macs without the traditionally expensive and arduous need to redevelop a game from the ground-up. Cider acts as a “wrapper” around the PC game dynamically translating PC API calls to the Mac OS X operating system. As such, games can be enabled with Cider in a matter of days to weeks as opposed to the typical man years that traditional development takes.”

Why use Cider? (or: what’s the point?)

Cider is often used as a too-little-too-late correction for a major mistake made in early development: the use of Windows-only APIs like .NET, Direct3D, DirectSound, GDI, etc. Programs written to use these APIs can’t run on platforms other than Windows without the use of an emulation layer.

The .NET Framework – Strictly speaking, i don’t find a lot of utility in the .NET framework and tend to use other tools that the framework was meant to simplify. However, it is a best-option in a lot of circumstances for game developers. Portability is probably the biggest thing to this. By “portability”, we speak strictly of Microsoft-only systems; desktop and handheld devices, but the big deal here for the gaming industry is the Xbox portability. Since it’s a gaming platform, it makes it possible to have one’s product be both a PC game and a console game without much additional overhead. Compare this to the Wii method which includes a separate system the size of a 4U server.

DirectX – DirectX is a multitude of utilities from complex graphic rendering to control handling (keyboard, mice, and game control pads) to sound and music playback. Along with the .NET framework, DirectX is used because it works on Window 95 and above, Windows CE (to a limited extent), and most importantly, the Xbox.

Who uses Cider? (or: Dear God, why?)

At this point, game companies, rather exclusively, make use of Cider. You might know a few games: The Sims 3, Spore, City of Heroes and EVE Online to name a few.

Cider’s aim is to make porting a Windows game to Mac a lot easier, eliminating many hours of coding time. For instance, DirectX isn’t supported on Mac OS X, so the graphics API would need to be remapped to use OpenGL. Generally speaking, Mac games from major distributors don’t use Cocoa or Quartz, but rather OpenGL. The reason for this is simple: OpenGL works on pretty much anything. Mac OS X, Linux, FreeBSD, Windows, the iPhone, etc.

By why use DirectX at all if compatibility is an issue? There’s a few reasons. Several years ago, DirectX was a lot better in terms of performance than OpenGL. As a result, several companies spent good sums of money to train their developers to use DirectX Instead of switching to OpenGL, companies generally avoid the extra cost of training and continue to use DirectX. Once a game is done, it’s passed along to a separate team that does the port. Because all the code is in place as far as what a function call can do, it’s basically a codeswap; DirectX calls translated to OpenGL calls. Of course, it’s a lot more complex than this, but this is the general idea to the average layman; consider it like a game with a story written in English being translated to Japanese.

The process really is a long one, however, so a lot of money and hours need to go into it.

So what’s the problem? (or: The Domino Effect)

Know the saying “you get what you pay for”? It doesn’t apply here. Mac gamers pay as much as Windows players, and find that their experience is really diminished. That’s because Cider runs as an emulation layer.

There are various types of emulators. Full hardware emulators (i.e. Bochs, QEMU, etc), virtualizers (i.e. VMware, VirtualBox), and API wrappers (i.e. Wine, Crossover, Cider, etc). Full hardware emulators deal with translating machine code between two completely different architectures. This is perhaps the most complex and painful part of emulation. It causes tremendous overhead unless it’s done very well (i.e. a combination of dynamic recompilation and jitting). In the case of an API wrapper, the overhead shouldn’t be as much. The hard work is already done. The code is already native (x86 or x86_64).

Unfortunately, nobody has yet done an API wrapper particularly well. Current API wrappers such as Wine run much slower than they should. A majority of the Windows APIs can be mapped to POSIX API calls without much effort. Windows threads become pthreads. Windows critical sections become pthread mutexes. DirectX calls become OpenGL calls. But the overhead of an API wrapper such as Cider is much more tremendous than one might expect.

Is there much of a performance impact? Oh yes. Cider doesn’t care if you have more than one processor. Your state-of-the-art graphics card will be chugging along as if it were a 3-year old model. OpenCL, which is coming out in 10.6? Worthless. The Physical Address Extension (PAE) that allows you to run on more than 4GB of RAM? Likely going to be ignored. That’s to name just the hardware limitations.

On top of this, there are software issues. The method of coding to make DirectX fast isn’t the same method as to make OpenGL fast. It’s going to run slower because the original code for the game is totally unmodified. Additionally, and probably the biggest hit to performance, is the translation of the common WinAPI functions, such as page fault handling. Page fault handling is a very basic need; it’s not unusual to see a page fault delta of 20,000 per second on a very active application (i.e. games, which is what this is all about!).

Transgaming, the company that makes Cider and Cedega, have based their products on Wine for Linux. Since Transgaming is making these products with gaming use in mind, we might expect to see that Cedega outperforms Wine. Let’s run some basic micro-benchmarks and see how Cedega and Wine perform compared to native execution on Windows XP. Tests with the largest differences are shown in bold (thanks to Darek Mihocka for the great micro-benchmarking tool!).

Windows XP on a 2.33GHz Intel Core 2 Duo:

test 1 os crit 2 : 237 ms, 79000 ps/instr, 12 MIPS, 184.3 clk
test 2 os try ok : 11 ms, 3666 ps/instr, 272 MIPS, 8.5 clk
test 3 os pg flt : 71 ms, 2366666 ps/instr, 0.422 MIPS, 5528.4 clk
test 4 os virtqr : 14 ms, 466666 ps/instr, 2 MIPS, 1089.1 clk
test 5 os virtpr : 16 ms, 533333 ps/instr, 1 MIPS, 1244.2 clk
test 6 os isbadr : 150 ms, 5000000 ps/instr, 0.200 MIPS, 11665.0 clk
test 7 os isgood : 47 ms, 15666 ps/instr, 63 MIPS, 36.5 clk
test 8 os vxpflt : 71 ms, 2366666 ps/instr, 0.422 MIPS, 5528.4 clk
test 9 CmpExch 2 : 214 ms, 71333 ps/instr, 14 MIPS, 166.4 clk
test 10 GetInpSt : 9 ms, 3000 ps/instr, 333 MIPS, 6.9 clk
test 11 GetQStat : 1186 ms, 395333 ps/instr, 2 MIPS, 922.4 clk
test 12 PeekMsg : 1377 ms, 459000 ps/instr, 2 MIPS, 1071.1 clk
test 13 MemMap 2 : 107 ms, 3566666 ps/instr, 0.280 MIPS, 8332.1 clk
test 14 QueryPrf : 5243 ms, 1747666 ps/instr, 0.572 MIPS, 4078.6 clk

Linux running Wine v1.0.1, on the same 2.33 GHz Intel Core 2 Duo:

test 1 os crit 2 : 295 ms, 98333 ps/instr, 10 MIPS, 229.4 clk (1.24x slower)
test 2 os try ok : 12 ms, 4000 ps/instr, 250 MIPS, 9.3 clk (1.09x slower)
test 3 os pg flt : 163 ms, 5433333 ps/instr, 0.184 MIPS, 12679.3 clk (2.29x slower)
test 4 os virtqr : 44 ms, 1466666 ps/instr, 0.681 MIPS, 3425.8 clk (3.14x slower)
test 5 os virtpr : 91 ms, 3033333 ps/instr, 0.329 MIPS, 7091.1 clk (5.68x slower)
test 6 os isbadr : 156 ms, 5200000 ps/instr, 0.192 MIPS, 12151.0 clk (1.04x slower)
test 7 os isgood : 71 ms, 23666 ps/instr, 42 MIPS, 55.2 clk (1.51x slower)
test 8 os vxpflt : 173 ms, 5766666 ps/instr, 0.173 MIPS, 13485.5 clk (2.43x slower)
test 9 CmpExch 2 : 217 ms, 72333 ps/instr, 13 MIPS, 168.7 clk (2.43x slower)
test 10 GetInpSt : 34990 ms, 11663333 ps/instr, 0.085 MIPS, 27447.0 clk (3887.78x slower)
test 11 GetQStat : 34967 ms, 11655666 ps/instr, 0.085 MIPS, 27447.0 clk (29.48x slower)
test 12 PeekMsg : 42974 ms, 14324666 ps/instr, 0.069 MIPS, 33811.5 clk (31.20x slower)
test 13 MemMap 2 : 2545 ms, 84833333 ps/instr, 0.011 MIPS, 212090.9 clk (23.78x slower)
test 14 QueryPrf : 4723 ms, 1574333 ps/instr, 0.635 MIPS, 3674.0 clk (1.12x faster)

Linux running Cedega 7.3.1 on the same 2.33GHz Core 2 Duo:

test 1 os crit 2 : 286 ms, 95333 ps/instr, 10 MIPS, 222.4 clk (1.21x slower)
test 2 os try ok : 24 ms, 8000 ps/instr, 125 MIPS, 18.6 clk (2.18x slower)
test 3 os pg flt : 1995 ms, 66500000 ps/instr, 0.015 MIPS, 155533.3 clk (28.10x slower)
test 4 os virtqr : 7 ms, 233333 ps/instr, 4 MIPS, 544.4 clk (2.0x faster)
test 5 os virtpr : 96 ms, 3200000 ps/instr, 0.312 MIPS, 7477.5 clk (6.00x slower)
test 6 os isbadr : 1975 ms, 65833333 ps/instr, 0.015 MIPS, 155533.3 clk (13.16x slower)
test 7 os isgood : 102 ms, 34000 ps/instr, 29 MIPS, 79.3 clk (6.00x slower)
test 8 os vxpflt: FAILED!! (games using vectored exception handling would crash)
test 9 CmpExch 2 : 289 ms, 96333 ps/instr, 10 MIPS, 224.7 clk (1.35x slower)
test 10 GetInpSt : 4925 ms, 1641666 ps/instr, 0.609 MIPS, 3830.8 clk (547.22x slower)
test 11 GetQStat : 4796 ms, 1598666 ps/instr, 0.625 MIPS, 3732.8 clk (4.04x slower)
test 12 PeekMsg : 41978 ms, 13992666 ps/instr, 0.071 MIPS, 32859.1 clk (30.49x slower)
test 13 MemMap 2 : 2983 ms, 99433333 ps/instr, 0.010 MIPS, 233300.0 clk (27.88x slower)
test 14 QueryPrf : 4078 ms, 1359333 ps/instr, 0.735 MIPS, 3174.1 clk (1.09x faster)

Oddly enough, the use of Cider is actually worse than using the open-source Wine. The result: a far less enjoyable experience, but probably more enjoyable than, say, running a Windows game in Parallels or VMWare Fusion.

In short, you get what the game developer paid for. Low performance for a low price.

If you’ve play Spore on the Mac platform at all (which I have and I’m throughly unimpressed with the game), then you’re probably thinking that I’m taking things out of proportion. Well no, I’m really not. Games like Spore and the Sims 3 aren’t exactly what we call graphics intensive, but games like EVE Online have the potential for very intensive sequences (namely battles). Here’s the bottom line: you’ll never find games like Bioshock, Crysis or even Portal running on the Mac (more on Portal later).

Should I be concerned? (or: So what’s the problem exactly?)

The use of Cider drops the bottom line. Little additional development is required and the price tag is a lot less than porting to native code.

Should you be concerned? Oh yes. Cider encourages developers to be lazy in the porting process, allowing them to exploit the Mac and Linux gaming markets with underachieving products.

In general, Macs can be good gaming platforms, specifically due to the hardware used in Macs on a given day. Not the best hardware, but certainly among the top. If you’re going to run a Windows game in BootCamp, then you’re going to be pretty well off on high detail modes, even on the laptop models (at least the newer ones). Using a Cider port, however, means that you might be getting intermittent frames, even on low detail modes.

This isn’t really a big deal though if you don’t play games, right? I beg to differ. Games are fairly complex in the emulation layer support they require. The main reason that we’re seeing this behaviour in games is because it’s the one industry that lacks porting the most.

It’s fairly safe to say that constructing a wrapper for other applications would be a lot easier, particularly if the wrapper is not a lot more than Wine. Imagine if other corporations started adopting the Cider method of porting. What could we see?

Imagine Adobe Photoshop or Premiere ported with a Cider wrapper. Or Autodesk Maya, Blender or Bryce. All of these have ports in native code, but what if, in the recessed economy, these companies decide to go the Cider route to cut the bottom line?

This would be a very good way to effectively ruin the Mac platform. While more apps could theoretically be used on Mac OS X, they would all be running in an emulation layer, cutting back on performance. If you use these utilities for a job, there’s a good chance that you would think twice before buying a Mac due to the performance issues.

Apple is trying to make a habit out of optimization. It’s the key focus of Snow Leopard. Unfortunately, no amount of optimized code on the part of Apple will matter if companies start favouring the Cider method outside the gaming industry. Already, gamers that were originally excited to hear of big names like EA coming to the Mac platform are instead feeling disenfranchised rather than empowered. Now imagine if this was happening in the video industry or the graphical arts industry.

It’s been said by some others that Cider could means the death of Mac gaming as we know it, but the potential here is so much more dangerous. It also has lasting implications.

Unintended consequences (or: Piracy Made Easy)

I stated before that I was going to get back to Portal, and now I am. There has been a Cider port of Portal running around for some time now, but it’s not based from Valve.

It is in fact a product of a new fresh mode of piracy. The advent of Cider has actually done more for the piracy movement than anyone else. Generally speaking, pirates understand that they’re already getting more than they are paying for, so certain graphical issues (some things won’t render, namely shaders) aren’t that big of a deal.

However, Portal and Half Life 2 are both critically acclaimed and are on many lists for people as a “must play at least once”. It would thus make sense that they would be targets for piracy. With Cider, that which wasn’t possible without a name brand virtualizer is now possible to run on the Mac, albeit without the quality that you would expect from a natural port.

Pirates don’t care. Free is free and it’s hard to complain when a cracked product is perfectly playable, even if it’s not pristine in graphical quality.

Afterword (or: Digression)

Cider really has a lot going for it in the future, and that’s inescapable. Being able to make any graphically dependent program truly cross platform is really no small feat, and the near-guarantee that a product can work on multiple platforms without significant code changes is a rather tempting reason to use it.

This however really brings to bear the use of the term “cross compatible”. A program that requires a virtualizer certainly does fall under the definition, but something like Cider isn’t so easy to classify. It’s not really even a grey area; it’s more like a splotchy murk. I worry that there might be a point in time that it start affecting the work I do. I rather enjoy the Mac platform, but I’d be damned if I use it to run a dozen wrapped applications instead of a dozen applications with native code.

I mean, the Mac version of Word is bad enough as it is.

I decided my desktop’s current appearance was worth sharing. Enjoy.

My Mom recently complained to me that her Hewlett-Packard brand PC was booting awfully slowly. She blamed it on my actions, claiming that it had to be the copy of Safari that I installed. She’s horrified to install anything on her computer for fear that it will “screw it up” or “slow it down”. In her case, it’s not a terribly unreasonable fear. If something does go wrong, she doesn’t know what to do to fix it. But blaming Safari for this problem did not make sense in the slightest, so I decided to take a peek under the hood and see why Windows Vista would boot so slowly on such a decent computer (besides the obvious, of course).

It took a while to find the Windows Vista equivalent to Windows XP’s BootVis tool, but apparently they made a new set of utilities for the task, called the Microsoft Windows Performance Toolkit. I won’t go into the usage of it, because you can Google it if you need that information. But I will show you how to interpret the results you get from it, and expose the culprit of this particular performance nightmare.

The first run showed a pretty horrendous sight:

Unfortunately the first results were rather worthless, as they were absolutely littered with disk input/output caused by AVG:

So I uninstalled AVG and ran the test a second time. This time, I got some conclusive results:

There are a few things to note about this graph. First of all, the core system boot sequence is shown to be taking 75 seconds (by “core system” I mean the time it takes between BIOS and loading a login screen). This is horrifying. Consider for a moment that Linux can finish the system boot in less than 20 seconds and reach a fully logged in graphical user interface in an additional 5-15 seconds. So in perspective, a total boot time of roughly 190 seconds (75 system + 115 userspace) is absolutely terrible.

It’s time to figure out the proximate cause. I noticed in the graph above that there is roughly a 20 second period during the system boot where there is minimal CPU usage, very few disk I/O requests, but the disk utilization is maxed out. This tells me that it’s doing a lot of slow requests which stall the system. I decided to look at the disk access graph, which shows where the drive is physically reading/storing data at a point in time.

Criminy. That 20 second stall (highlighted in the second image above) is being caused by thrashing between some files that are physically very far apart on disk. Here’s a zoom on that section:

Well, details are necessary now. We need to know what files it is thrashing between (and why) in order to be able to do anything about it.

Aha! Success! Er, well, maybe not. The files are the “COMPONENTS” and “SOFTWARE” registry hives. This is pretty much bad news because the Windows defragmenter can’t resolve this problem for two reasons:

It’s very strange though, that Windows didn’t simply read the two files in their entirety in two large reads instead of this silly thousands-of-tiny-reads crap. I think this could very well be a bug in Windows Vista. Come on, Microsoft. What were you thinking? A programming 101 student could tell you that a large disk read is faster than lots of smaller ones.

So in order to work around the problem, I first needed to be able to defragment my Mom’s computer’s hard disk completely without running the installed operating system. It’s unfortunate she doesn’t have a Mac, because if she did, we could simply use Target Disk Mode which basically makes the computer behave like an external FireWire drive; all that would be necessary in such a case is hooking it up to another machine as you would with an external hard drive, and then defragment that way. Unfortunately for her, the easiest way to do this was to repartition the drive so that we could install a temporary copy of Windows in a second partition.

The second issue is that we needed to be able to optimize the drive, and not just defragment it. By “optimize”, I mean physically arranging files on disk in such a way that they’re quickly accessed during the system’s boot sequence. This is different from defragmentation, which simply consolidates split files. Thankfully, there’s an awesome open source defragmenter available called JkDefrag, which is capable of optimizing as well as defragmenting. Sweet!

So I installed Windows 7 RC1 onto another partition (created via Disk Management), and then ran JkDefrag with the command-line option ‘-a 7′, which tells JkDefrag to optimize by arranging the files alphabetically based on their path and file name. This should cluster the contents of folders together, making accesses much faster.

So after a couple hours, it finally became bored and finished defragmenting, so I rebooted to re-time Windows Vista’s boot sequence. It was noticeably faster already, but I needed numbers. And I was quite pleased with the results:

Note that there’s no obvious point at which a bottleneck is noticeable. Neither the CPU nor the hard drive are maxed out for too long at any one point. And then there’s the physical layout access graph:

Awesome. JkDefrag did its job and the files are much better clustered. Better yet, the raw numbers are even more impressive. The system boot has been reduced to 45 seconds, which is 60% of the time it took previously. Based on the disk and CPU activity, the machine’s boot is complete after about 90 seconds, which is 47% of the time it took previously. (Keep in mind that xbootmgr continues measuring for 120 seconds after the user logs in, so we can’t just use the last time marker on the boot graph as the time it takes to boot. You have to look at disk activity and CPU utilization, and figuring out the point where the constant activity ceases and the machine begins to settle.)

So now my mom’s machine boots pretty quickly. Now I need to move on to my next project: Operation: Get Mom to Stop Using Internet Explorer.

Well, that was frustrating. I stayed up late last night trying to figure out why my Apache 2.2 server was misbehaving, and it ended up being a Firefox 3 bug. Or more specifically, a bug in the old OpenSSL library that Firefox 3 is statically linked to.

I tried to view our cgit page, and only got this:

And if I refreshed a few times, I got this ’ssl_error_rx_unexpected_change_cipher’ error:

But strangely, if I used Safari, I got exactly what I should have:

I finally figured out that Firefox was screwing up when using the TLS 1.0 protocol. So all I had to do was edit my Apache 2.2 httpd.conf to have this line:

SSLProtocol -SSLv2 +SSLv3 -TLSv1

I don’t like this solution though. The Mozilla Firefox team should release a build of Firefox that is statically linked to a newer version of the OpenSSL library which doesn’t suffer from this bug. I much prefer TLS because of the added security it provides, so I ended up with a line that opted for security over broken SSL library compatibility:

SSLProtocol -SSLv2 -SSLv3 +TLSv1

Also note that I disable SSL v2 as well bacause it’s widely known to be inherently flawed.

Well, I just finished Fallout 3. It’s an interesting game, and I greatly enjoyed a large portion of it. I especially appreciate how Bethesda built Fallout 3 on the same game engine that Oblivion is based on. Some things, however, really bug me.

One, a majority of the game dialogue is bland and unexciting. This is particularly bad for the stat-checked dialogue. For instance, there’s one marked “[Intelligence]“, which is supposed to be a point in which you impart some keen insight. Unfortunately, the “insight” is actually quite weak, and tends to merely restate the obvious. A tag like “[Captain Obvious]” would be far more true to the actual insight given. Another problem with the dialogue is that non-player characters are so easily swayed. A non-player character could be steadfastly against something, but as long as your Speech skill is high enough, you can convince them to jump off a bridge. Dialogue that involves convincing an NPC about something goes something like this:

Player: You should do this.
NPC: No! It goes against the way I have been brought up, and against everything I believe!
Player: [Speech 100%] You really should do this.
NPC: OK.

I wish this was a joke, but it really is this bad.

Two, the open-endedness seemed far too limited. In TES IV: Oblivion, you could finish the main quest and still have many, many hours of enjoyable gameplay left, largely because of the sheer number of quests. Not so in Fallout 3. You finish the main quest and it’s game over, regardless of whether or not your character does the noble self-sacrifice thing.

I would have liked a better conclusion to the Trouble on the Homefront quest, which really is not very well written. In the way I ended the quest, the player character’s lifelong friend Amata becomes Overseer of Vault 101. Unfortunately, her character does an about-face and basically takes on the personality of her father (the previous Overseer), and exiles the player from Vault 101 forever. I would have liked to see a better conclusion to this quest. Amata’s stated goal, before her character’s about-face, was to open the vault for trade and so forth. With the way the quest ends, she closes the vault for the remainder of the game. It would have made much more sense to me that Vault 101 would be closed for an in-game week or two, to allow her to restore order and bring the vault back to its former state, and eventually open, allowing the player to revisit the vault, old friends, etc. It would also have been more interesting to see residents of Vault 101 visiting Megaton or other nearby places for supplies. Other people have noted their disappointment for their results for the quest as well.

Despite these negative points, I found Fallout 3 to be generally enjoyable. The graphics were superb and the landscape was exactly what I would expect from a post nuclear holocaust environment. The sounds were nicely done as well, and it’s nice to see that Bethesda got more than a handful of voice actors for Fallout 3.

I think Fallout 3 had a lot more potential, and I’m hoping that the downloadable content packs will add something substantial to the game, unlike, for instance, this.

I just installed service pack 1 on a 64-bit Windows Vista machine, and it stupidly took forever and a half to install. It also took multiple reboot cycles, something that wasn’t needed in previous Windows NT operating systems (2000 and XP included). I think Microsoft’s going the wrong direction with Windows service packs, because it is now ridiculously hard to integrate into a customized Windows disc. With Windows XP service packs, it was simple. You’d copy the files to the hard drive, slipstream, and burn the disc. The whole process took about 20 minutes. With Vista, it’s an interminable procedure requiring third party software (vLite). The process takes an hour or more.

Another service pack that I’ve had annoying troubles with is Visual Studio 2008 SP1. It takes more than an hour to install. In fact, it’s still installing on this machine. I wouldn’t be surprised if it’s actually stuck, because the progress bar hasn’t moved in 20 minutes or so.

Why the hell do Visual Studio service packs take forever to install? I assume the problem with 2008 SP1 is similar to the 2005 SP1 issue, which required a very ugly hack to get around it. I mean, it works, but it just takes forever. With Visual Studio 6, you could install a service pack in less than 5 minutes. This process has ballooned to over an hour. Why?!

Another thing that’s bugging me is that Visual Studio 2008 feels slower than molasses. Apparently I’m not alone in noticing this.

Microsoft’s stuff is just seeming less and less appealing. I can get a decent development machine running Mac OS X set up in about an hour or maybe two (including Xcode, Git, development libraries [SDL, etc], CMake, Smultron, etc). On Windows, It’s taken me at least 8 hours so far. I’ve been installing Windows, Visual Studio, service packs and other updates constantly. And I mean constantly. I haven’t let the machine idle once.

Microsoft! Give up on all this stupidity you’ve been adding to Windows, Visual Studio, and Office for Mac! Slim your applications. Make performance and size a priority. To ignore performance will be an absolutely fatal mistake for you. The only teams I think really have it together right now are the Microsoft Office (Windows) team, and the Xbox team. Their stuff installs easily, updates easily, and runs smoothly. Why can’t the rest of Microsoft get this crap right?!

As a software developer, I find it important to ensure that my programs work well on all platforms. Windows, Mac OS, Linux, whatever. So I decided I should triple-boot my MacBook Pro instead of juggling between a Mac/Windows or Mac/Linux setup.

I’m annoyed that every article I’ve found about setting up a Mac to triple-boot makes it sound like it’s impossible to have greater than four partitions (especially for triple-booting). This simply isn’t the case, and it is hazardous for Linux’s performance in particular because you’re forced to set up a swap file instead of a swap partition and in the case of Gentoo, the /usr/portage mountpoint can’t be separate from the main filesystem (/usr/portage changes frequently and quite randomly, generating awful fragmentation). I’ve had Gentoo installs which had 8 partitions just for Gentoo itself (split up the filesystem to better optimize the usage).

In the past, the problem has been that Intel Macs use the GUID Partition Table (GPT) scheme for representing partition information on disk. Since Windows even now only recognizes the Master Boot Record (MBR) format, Apple came up with a trick that basically makes the disk use a GPT/MBR hybrid. It stores two copies of the partition table. One in GPT format, and one in MBR format. Seems straightforward? It gets tricky.

With MBR, you are limited to 4 ‘primary’ partitions. Once MBR was designed and out the door, people began to realize that 4 partitions just wasn’t enough, and an extension to MBR was developed that would not break compatibility with the existing MBR definition. The extension allowed for one of the primary partitions to be marked as an ‘extended’ partition. Extended partitions sort of store their own partition table which keeps track of the ‘logical’ partitions contained in them.

With the GPT/MBR hybrid, Mac OS X will synchronize the tables when the MBR doesn’t match the GPT table. Unfortunately, GPT doesn’t understand extended/logical partitions and Apple’s sync tool refuses to sync when it detects the MBR has extended partitions defined. So the limitation here is that we cannot have any extended partitions in the MBR if we want GPT/MBR syncing to work.

The solution is pretty simple, really. Windows requires MBR partitioning, but the Linux kernel does not. Simply set up the partition tables with gparted, run the gptsync tool (which can be run pretty easily if you have rEFIt), install Windows, and finally install Linux. Windows will only see the MBR partitioning scheme, and if your Linux distro supports GPT, Linux will see the GPT scheme. Detailed steps are below:

1. Install Mac OS X. One partition spanning the entire disk for now.
2. Run the Boot Camp Assistant and allocate however much space you want for Windows/Linux. I allocated roughly two thirds of the drive. When you get to the last step, just say ‘Quit’ instead of ‘Start install now’.
3. Get a Linux distribution with a LiveCD that supports GPT. There are quite a few, but off the top of my head, I can only say with certainty that Ubuntu’s LiveCD supports this.
4. Boot the Linux LiveCD and run gparted
5. Boot Camp sets up a solitary partition for Windows; wipe it out. Add a 100MB partition (this will later be an ext2-formatted /boot partition), an NTFS partition immediately following this, and then whatever other partitions you need (for Ubuntu, you should have /, /home, and a swap partition).
6. Write the partition tables to disk when you’re satisfied. If your gparted is too old, it will trash the MBR. It’s not terribly bad news. Just run the GPT sync tool (detailed below), and then continue from the next step.
7. Now bring up a terminal, run ‘cfdisk’, and mark the NTFS partition bootable. Write that table to disk. Reboot.
8. Assuming you have rEFIt installed, it should show the rEFIt bootloader. Run the partition table sync tool found on this screen. Make sure that the settings it shows are right. It will show that the new MBR will only have 4 partitions listed, but that’s fine. If things look peachy, tell it ‘y’ and hit enter. Now turn the machine off. I have found that if you use the sync tool or anything EFI related before selecting something to boot, the system will just sit there and hang. It’s not destructive; it’s just annoying.
9. Go ahead and install Windows on the NTFS partition. Things should go fine. I assume you have used Boot Camp before, so you can do your Windows driver installs and updates, etc. on your own.
10. Now install Linux. In the case of Ubuntu, don’t let it auto partition. Select ‘Manual’ when it asks about it. Tell it what you wanted the partitions to mounted as (100MB is /boot, etc). Once ready, click Next. Once you get to the summary screen that tells what the installer is about to do, click ‘Advanced’. The field which indicates where GRUB will be installed needs to be altered. You don’t want to alter the disk’s MBR, you just want to modify the boot sector on the /boot partition. Figure out the partition name (in my case, /dev/sda3), and put that in there. This ensures that you can boot Windows, Mac and Linux directly from rEFIt instead of being forced to use GRUB for both Windows and Linux.
11. You should now have a Mac+Linux+Windows machine. Great for software development!

I kind of whipped this article up in a hurry. If there are any questions about this, email me.

Have you seen Microsoft’s latest act of desperation aiming to save Windows Vista?

It’s interesting to me that Microsoft has resorted to this strange act. I mean, I knew Microsoft was bleeding money, but I had no idea they were this desperate. You might wonder how this is an act of desperation. Well, Microsoft has taken several non-technically-minded people and asked them to criticize an operating system with technical flaws. As I said back in December, Windows Vista has some glaring flaws that a technically-minded person would pick up on fairly quickly. Unfortunately, all the end users see is the pretty new face, mainly the Aero Glass interface, and think it’s fantastic.

To me, it would have made much more sense to correct the technical flaws, implement the changes in Windows 7 and begin hyping it up. But of course, this isn’t really a new marketing theme, this is truly an underhanded act of desperation.

I was doing a bit of research not long ago, looking for a Windows XP equivalent of Windows Vista’s ReadyBoost feature. As it turns out, there’s an awesome product available called eBoostr which does what ReadyBoost does and more.

ReadyBoost is perhaps one of the few truly nifty improvements in Windows Vista: the ability to add an additional layer of RAM to your machine without actually adding any RAM modules. The idea is fairly simple, but before I get into that, a brief primer on the various types of memory in a computer system.

The fastest memory in the computer is the cache built into the CPU itself. This memory is very tiny and expensive to create, but it’s blindingly fast. The next layer of memory is RAM, which is slower than the on-die processor cache, but larger and cheaper to create. Typically, the final layer of memory is the hard disk. The hard disk is significantly slower than RAM (RAM is about 50x faster in most cases), largely because of the necessity to seek to a location on the disk in order to read data. The hard disk is also non-volatile storage. A power loss does not eliminate the data contained on the drive, unlike in the cases of the CPU cache and RAM chips. In one of the machines I’ve got here, the speeds are as follows:

• CPU L1 cache (64KB) 11-12 GB/s
• CPU L2 cache (2048KB) 6-7 GB/s
• System RAM (2048MB) 2-3 GB/s
• Hard drive (100GB) 20-40MB/s

Now that you have an idea of how the speeds typically run, I should explain what ReadyBoost does. ReadyBoost allows you to add a new layer of memory, between the system RAM and hard drive layers in the list above. With ReadyBoost, you can turn any reasonably fast thumbdrive into a disk cache. Common reads from the hard drive can be stored in the thumbdrive’s memory. For instance, when you start your web browser, you’re touching dozens of different files (the browser’s main executable, any DLLs it depends on, etc). If you run the program enough, it will be cached on the ReadyBoost-managed thumbdrive so that the next time those files are accessed, they can be read much much quicker than if they were being read from the hard drive itself. This also makes it so that the hard drive is available for other read/write operations and eliminates a lot of seek time and read time.

eBoostr kind of beats the pants off of ReadyBoost, though. One of eBoostr’s advantages is the ability to use more than one cache device at a time, but there’s an even bigger advantage. Not only can you use a thumbdrive as a disk cache, but you can also use extra system RAM as a disk cache. For instance, in my MacBook Pro, I’ve got 3GB of RAM. I very very rarely use up more than a gigabyte or maybe one and a half gigabytes, even when gaming. So what I have done with eBoostr is assign 1GB of it to act as a disk cache. The speed improvement is extraordinary! Typically, it is on the order of 10 to 100 times the speed of a disk read.

eBoostr also collects statistics about which files are accessed frequently to improve its caching mechanisms. I recently installed eBoostr on my mother’s laptop, which she uses frequently for predictable tasks. She runs Internet Explorer, Outlook Express, Windows Live Messenger, Solitaire, FreeCell, and a select few other apps quite frequently. She sparingly uses other programs, and eBoostr knows this. I looked at eBoostr’s cache hit rate on her machine, and it sticks at about 95% all the time. This means that for 95% of disk reads, they are taken directly from her RAM cache. For a machine like hers, this offers a 30x speed increase for those particular reads.

In my case, though, the statistical analysis that eBoostr tries isn’t quite as helpful. I have a fairly unpredictable usage schedule. I will use Outlook, Firefox, Skype, Cygwin, Visual Studio and various apps that come and go on my machine as I decide I do or don’t want them. My hit rate stays at about 85%, which is still fantastic.

I very highly recommend eBoostr for anyone running Windows XP. It’s a great product, and really should be supported!

Alright, I’d better give bit of technical info for suspicious geeks. I do realize that there are a ton of scam products out there which claim to “make computers faster”, but eBoostr is one of a handful of technically feasible products. The way eBoostr works is that it installs a system driver which hooks any sort of disk I/O. If there’s a write to a file that’s in the cache, eBoostr marks that item invalid and re-caches it on the next read of that file. If there’s a read, eBoostr checks if it’s in the cache, and if it is, it throws the cached file back at the program instead of allowing a hard drive read. If it’s not in the cache, eBoostr reads the file into its cache so that it is much faster to read the next time it’s accessed. Don’t worry about writes, though; they are not cached, because doing so would introduce horrid unreliability in the case of a power loss or system crash.

I don’t know whether to blame the hard drive, Windows 2000, or NTFS-3G. But never in my life do I want to see chkdsk show this again:

It kept scrolling that text for about 10 minutes before finally hitting this BSOD:

So we booted a Windows Vista DVD and ran chkdsk with that to finish the job it couldn’t:

Long story short, we were trying to reformat an external hard drive and ended up putting temporary copies of the data onto the local disk. Unfortunately, the local disk went haywire at that moment, and to make a long story short, most of the files were nicely toasted. The machine needed to be reformatted as well.