Surviving the RAM Apocalypse With Software Optimizations
To the surprise of almost nobody, the unprecedented build-out of datacenters and the equipping of them with servers for so-called ‘AI’ has led to a massive shortage of certain components. With random access memory (RAM) being so far the most heavily affected and with storage in the form of HDDs and SSDs not far behind, this has led many to ask the question of how we will survive the coming months, years, decades, or however-long the current AI bubble will last.
One thing is already certain, and that is that we will have to make our current computer systems last longer, and forego simply tossing in more sticks of RAM in favor of doing more with less. This is easy to imagine for those of us who remember running a full-blown Windows desktop system on a sub-GHz x86 system with less than a GB of RAM, but might require some adjustment for everyone else.
In short, what can us software developers do differently to make a hundred MB of RAM stretch further, and make a GB of storage space look positively spacious again?
Just What Happened?
On these systems I could run a browser with many tabs open, alongside an office suite, an IDE, chat applications like IRC and ICQ, an email client, filesharing applications, and much more, without the system breaking a sweat. In the Duron 600 system I would eventually install a Matrox G550 AGP videocard to do some dual-monitor action, like watching videos or consulting documentation while browsing or programming at the same time.
Fast-forward a few decades and you cannot even install Windows on a 1 GB partition, and it requires more RAM than that. A quick check on the Windows 10 system that I’m typing this on shows that currently the Windows folder is nearly 27 GB in size and just the Thunderbird email client is gobbling up over 150 MB of RAM by itself. Compare this to the minimum Windows 2000 system requirements of a Pentium 133 MHz, 32 MB of RAM and 1 GB of free HDD space.
This raises the question of what the reason is for this increase, when that email client in the early 2000s had effectively the same features in a much smaller package, and Windows 2000 is effectively the same as Windows 7, 10 and now 11, at its core when it comes to its feature set.
The same is true for ‘fast and light’ options like Linux, which I had once running on a 486DX2-66 system, a system on which the average Linux distribution today won’t even launch the installer, unless you go for a minimalistic distro like Alpine Linux, which requires a mere 128 MB of RAM. Where does all this demand for extra RAM and disk storage come from? Is it just all lazy waste and bloat that merely fills up the available space like a noxious gas?
Asking The Right Questions
Storage and RAM requirements for software are linked in the sense that much of an application’s code and related resources are loaded into RAM at some point, but there is also the part of RAM that gets filled with data that the application generates while running. This gives us a lens to find out where the higher requirements come from.
In the case of Windows, the increase in minimum storage space requirements from 1 GB to 32 GB for Windows 10 can be explained by something that happened when Windows Vista rolled around along with changes to WinSxS, which is Windows’ implementation of side-by-side assembly.
By putting all core OS files in a single WinSxS folder and hard-linking them to various locations in the file system, all files are kept in a single location, with their own manifest and previous versions kept around for easy rollback. In Windows 2000, WinSxS was not yet used for the whole OS like this, mostly just to prevent ‘DLL Hell’ file duplication issues, but Vista and onwards leaned much more heavily into this approach as they literally dumped every single OS file into this folder.
While that by itself isn’t such an issue, keeping copies of older file versions ensured that with each Windows Update cycle the WinSxS folder grew a little bit more. This was confirmed in a 2008 TechNet blog post, and though really old files are supposed to be culled now, it clearly has ensured that a modern Windows installation grows to far beyond that of pre-Vista OSes.
Thus we have some idea of why disk storage size requirements are increasing, leading us to the next thing, which is the noticeable increase in binary size. This can be put down for a large part on increased levels of abstractions, both in system programming languages, as well as scripting languages and frameworks.
Losing Sight Of The Hardware
Over the past decades we have seen a major shift away from programming languages and language features that work directly with the hardware to ones that increasingly abstract away the hardware. This shift was obvious in the 90s already, with for example Visual Basic continuing the legacy of BASIC with a similar mild level of abstraction before Java arrived on the scene with its own virtual hardware platform that insisted that hardware was just an illusion that software developers ought to not bother with.
Subsequently we saw .NET, JavaScript, Python, and kin surge to the foreground, offering ‘easier programming’ and ‘more portable code’, yet at the same time increasing complexity, abstraction levels, as well as file sizes and memory usage. Most importantly, these languages abandoned the concept of programming the underlying hardware with as few levels of indirection as possible. This is something which has even become part of languages like C and C++, with my own loathing for this complexity and abstraction in C++ being very palpable.
In the case of a language like Python, it’s known to be exceedingly slow due to its architecture, which results in the atrocious CPython runtime as well as better, but far more complex alternatives. This is a software architecture that effectively ignores the hardware’s architecture, which thus results in bringing in a lot of unnecessary complexity. Languages such as JavaScript also make this mistake, with a heavy runtime that requires features such as type-checking and garbage collection that add complexity, while needing more code to enable features like Just-In-Time compilation to keep things still somewhat zippy.
With Java we even saw special JVM processor extensions being added to ARM processor with Jazelle direct bytecode execution (DBX) to make mobile games on cellphones programmed in J2ME not run at less than 1 FPS. Clearly if the software refuses to work with the hardware, the hardware has to adapt to the software.
By the time that you’re a few levels of abstraction, various ‘convenient frameworks’ and multiple layers of indirection down the proverbial rabbit hole, suddenly your application’s codebase has ballooned by a few 100k LoC, the final binary comes in at 100+ MB and dial-up users just whimper as they see the size of the installer. But at least now we know why modern-day Thunderbird uses more RAM than what an average PC would have had installed around 1999.
Not All Hope Is Lost
There’s no need to return to the days of chiseling raw assembly into stone tables like in the days when the 6502 and Z80 still reigned supreme. All we need to do to make the most of the RAM and storage we have, is to ask ourselves at each point whether there isn’t a more direct and less complex way. What this looks like will depend on the application, but the approach that I like to use with my own projects is that of the chronically lazy developer who doesn’t like writing more code than absolutely necessary, hates complexity because it takes effort and whose eyes glaze over at overly verbose documentation.
One could argue that there’s considerable overlap between KISS and laziness, in the sense that a handful of source files accompanied by a brief Makefile is simultaneously less work and less complex than a MB+ codebase that exceeds the capabilities of a single developer with a basic editor like Notepad++ or Vim. This incidentally is why I do not use IDEs but prefer to only rely on outrageously advanced features such as syntax highlighting and auto-indent. Using my brain for human-powered Intellisense makes for a good mental exercise.
I also avoid complex file formats like XML and their beefy parsers, preferring to instead use the INI format that’s both much easier to edit and parse. For embedding scripting languages I use the strongly-typed AngelScript, which is effectively scriptable C++ and doesn’t try any cute alternative architectures like Python or Lua do.
Rather than using bulky, overly bloated C++ frameworks like Boost, I use the much smaller and less complex Poco libraries, or my NPoco fork that targets FreeRTOS and similar embedded platforms. With my remote procedure call (RPC) framework NymphRPC I opted for a low-level, zero copy approach that tries to stick as closely to the CPU and memory system’s capabilities as feasible to do the work with the fewest resources possible.
While I’m not trying to claim that my approach is the One True Approach
Sure, I could probably have done something with MicroPython or so, but at the cost of a lot more storage and with far less performance. Which gets us back again to why modern day PCs need so much RAM and storage. It’s not a bug, but a feature of the system many of us opted for, or were told was the Modern Way