Wednesday, October 5, 2011

Second attempt at PyPy

Update (Oct 30): The fourth translation failed due to insufficient memory again. I wonder if PyPy buildbot is a 32GB RAM machine ;).

A few days ago I made my first attempt at running PyPy. It was unsuccessful for my need due to bug #887. So today I gave it another try with a patch from Justin applied. To cut the story short, today was another failure. Read on for more long-winded story.

I checked out the source from BitBucket. Then apply the patch that was provided in the same issue above.

So, my first attempt was to use a pre-built PyPy binary to translate this source. It took about an hour or so before the process was killed. I was away when this happened so I did not know what caused the termination. Experience told me, though, that it might be due to out-of-memory behavior of Linux systems.

The second time I still used PyPy to make the translation. This time, I made sure to free as much memory as I could, and monitor its use. Another hour passed, the translation process once again was killed half way. Luckily, I had seen the cause clearly. Indeed, the translation process used too much memory and was killed forcefully.

Then I used the less memory hungry command to translate PyPy with pre-built PyPy. It consumed a little bit above 3 GB of RAM, enough to give me hope that it would succeed. However, before launching make -j 4 to compile generated C sources, translate.py did not release (or garbage collect) all of its unused objects. This left a waste of 3 GB and set aside only 1 GB (I was on a 64-bit CentOS 5.6 with 4GB of RAM) for GCC to do its work. Apparently, this wasn't enough and the same forced-kill story repeated.

Now I am in the fourth translation run. This time I do it with vanilla Python 2.6 instead of with pre-built PyPy. The process is indeed much slower, but it also consumes much less memory. The translation has run for an hour and a half and it is still running. Wish me luck.

No comments:

Post a Comment