Anomalous Anomaly

This week has been slightly heavier on schoolwork than usual (it is very close to the end of the quarter at Central Washington University). So most of my week has been spent working on tcmalloc and trying to figure out why the heap profiler is broken on Mac OS X. You can see the progress on the tcmalloc issue here. There’s a few embarrassing goofs on my part in there, but oh well.

As for Unladen Swallow, I’ve been looking at it in Instruments.

The above screenshot is of two runs of the ‘nbody’ test, which is from the Unladen Swallow test tree. The first run, top, is CPython 2.6.5 (I built my own because I didn’t want any Apple-specific code to interfere). The second run, right beneath it, is Unladen Swallow. ‘nbody’ was the test that had the highest memory usage increase when comparing Unladen Swallow to CPython. Interestingly, this memory usage appears to mostly just be a spike.

I suspect what happened in the above is that Unladen determined that the particular function it was running was considered “hot”, and then proceeded to JIT compile it. The reason it spikes is that compilation is taking quite a bit of memory, but after the end result is obtained, that memory isn’t needed anymore. There’re lots of possibilities for why there’s so much memory usage during that JIT compile. When I look at the objects which are created and still living between the time points I highlighted above, it looks like the overall memory usage increase is from 3.38MB to 5.71MB, an increase of only 2.33MB, which is acceptable. But between those two points, there is a grand total of 78.74MB allocated and freed. At its peak, the memory usage hits 13.71MB. This is where I need to focus my work. Why is it allocating so much for this particular test, considering other tests don’t cause such a large spike? How can I cut this down?

Observant readers familiar with Instruments will notice there’s a memory leak displayed in the screenshot above. This is a memory leak in LLVM 2.7, which has apparently been fixed between release and now, so it’s not particularly relevant.

I’m excited for when school is finally out. I’ll be able to really get going on this project!

And so begins my goal to help resolve Unladen Swallow‘s memory bloat issues. You can look at my proposal here.

Well, the week has gotten off to an interesting start. I’ve been trying to get Google perftools to work, so I can start doing some heap profiling on Unladen Swallow. Unfortunately, it’s not working nicely, even with one of my own projects. My first order of business for the week has been to fix problems I’ve found with perftools: #95, #244, #245, #246.

But I don’t intend to get too sidetracked by this. Perftools will definitely be an essential tool to have, since it allows for programatic, selective profiling. For the moment, I think I’ll just use the tool provided by Apple with Xcode called Instruments. It’ll suffice for the moment, but it doesn’t really have the fine-grained capabilities of perftools. My other tool of choice is Valgrind, which can be used for both finding memory leaks and heap profiling via massif.

Alrighty. Time to get cracking.

Miah suggested that I take a look at the server headers for Reddit.com, because he found something hilarious:

tycho@alcarin ~ $ curl -I http://www.reddit.com/
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Set-Cookie: reddit_first=%7B%22organic_pos%22%3A%201%2C%20%22firsttime%22%3A%20%22first%22%7D; Domain=reddit.com; expires=Thu, 31 Dec 2037 23:59:59 GMT; Path=/
Server: '; DROP TABLE servertypes; --
Date: Thu, 06 May 2010 23:09:17 GMT
Connection: keep-alive

I wonder what prompted them to put that in there…