Altman’s Memory Problem: The Importance of Being Idle
Your laptop’s memory has a job even when it isn’t working: being ready when you are. Sam Altman, CEO of OpenAI, the company behind ChatGPT, seems concerned about its work ethic.
Altman made the case during an Oct. 29 appearance on Cheeky Pint, the podcast hosted by Stripe co-founder John Collison. They were discussing why high prices hadn’t prompted memory manufacturers to expand supply faster. Altman challenged the idea that running artificial intelligence (AI) models on personal computers would ease the need for data centers. With memory scarce, he argued, centralized systems could put it to more consistent use.
Altman strikes me as earnest. I think he means it when he says that, from a “global allocation perspective,” it’s “quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer… being used 5% of the time.” DRAM—dynamic random-access memory—is the working memory computers use to run programs.
His premise is straightforward: Memory is scarce, so it should go wherever it gets the heaviest, most consistent use. By that logic, memory sitting idle in a laptop 95% of the time would serve us better in a data center, working almost continuously for thousands of users.
That view is understandable coming from someone whose firm needs vast data centers to run AI models. But it mistakes how often we use something for how much we value it. Assets we expect to use for years can justify their cost simply by being available when we need them. Privacy, control, and peace of mind also count, even if they don’t show up in a utilization rate.
More subtle versions of this mistake can be just as well-intentioned. In a recent interview with Ben Thompson, Palantir Chief Technology Officer Shyam Sankar aptly compared the current AI revolution to the spread of electricity in the late 19th and early 20th centuries. His central point is correct: Realizing the gains from this new technology requires redesigning how we work.
But the electricity analogy has limits. Altman’s complaint about idle memory shows where it begins to break down.