So recently I decided to scaled down even further in the cloud (well, webfaction is good but it’s shared, and many of the cloud services online are just expensive [Azure's cheapest with 1GHz dedicated core and 768MB RAM costs around USD$9 a month, while AWS's crappy micro instance throttles my server to a standstill regularly]), and I found this. Got their $19/year plan, and it comes with 192MB RAM, burstable up to 220MB, with CentOS 64bit.
I got one, and all was well, that was, at least before I ran yum update (note, i tested on ubuntu and they had the same problem, so stop saying it’s a red-hat issue =P). The second I ran yum update, tons of malloc/not-enough-memory errors started popping out. So, first lesson of the day, disable yum’s fastest mirror plugin (edit the file /etc/yum/pluginconf.d/fastestmirror.conf and set enabled=0).
After you do that, if you try updating again, you’ll run into bigger problems. The installation will crash halfway, and before you know it, you will have 2 version of everything on your machine, and fixing it requires you to remove the stuff one by one. The fix for this is to limit the stack size in the operating system. Set your maximum stack size to the amount of RAM you have (ulimit -s 192), and you will be fine. I put it in my /etc/rc.local file to ensure that the limit was always there when scripts were being run.