Thought Leadership
In view of scarce memory chips, the OpenAI chief considers it better to run AI centrally in data centers rather than locally on private computers.
OpenAI CEO Sam Altman considers it poorly utilized resources when AI models run locally on private computers. He said this on the podcast “Cheeky Pint” by Stripe co-founder John Collison. The occasion was the question of whether users will run open-source models on their own or custom-built computers in the future.
“From the perspective of global distribution, it is actually pretty bad if everyone has a portion of the globally limited supply of DRAM in their own computer that is only used 5 percent of the time.”
Sam Altman, OpenAI CEO
In an environment with severely limited supply, one wants the highest possible utilization in the cloud. However, the intuition of many people is the opposite.
Business model with models in the cloud
While OpenAI also offers some open-weight models, it earns its money primarily with closed models that run in data centers and are paid for via subscription or usage. Locally run models could diminish these revenues in the long term if users operate capable models on their own hardware. In October, Meta released Muse Spark Open, an open-weight model with 816 billion parameters optimized for local operation on two Mac Studios.
Memory prices more than doubled
The background is the ongoing shortage of memory chips. According to Gartner, prices for memory have more than doubled in the past twelve months because manufacturers are aligning their production toward AI data centers. Gartner expects this effect to persist until 2027. PC manufacturers and retailers are responding with higher prices; worldwide PC shipments fell by 20.1 percent in the third quarter, according to IDC.
However, hardware is not continuously utilized in data centers either. According to a survey by VentureBeat Intelligence among companies using AI productively, 79 percent of respondents reported that their GPUs are busy at most half of the time.
(Editorial Team)
