OpenAI’s Struggle with GPUs Exposes Big Tech’s Dominance in AI Market

• OpenAI CEO Sam Altman admits that the company does not have enough GPUs to meet the increasing demand for ChatGPT.
• This is due to the soaring demand for GPUs, which are essential for training and running AI models.
• Big Tech companies like Microsoft and Amazon have a significant advantage in the AI infrastructure market due to their direct access to GPUs and established customer bases.
• Smaller AI companies are facing an uphill battle as they lack the resources to compete with Big Tech.

KEYPOINTS

• The AI infrastructure market is consolidating, with Big Tech companies solidifying their power.
• Smaller AI companies are struggling to compete due to the high cost of GPUs and cloud services.
• There is some hope for smaller companies as the chip shortage eases and competition between cloud providers intensifies.

In a surprising revelation, Sam Altman, the CEO of OpenAI, admitted during a recent discussion with US senators that he would prefer users to rely less on ChatGPT, their popular language model. The reason behind this unexpected statement? OpenAI simply does not have enough GPUs to meet the increasing demand. Altman’s admission sheds light on a concerning trend in the generative AI industry, where established tech giants are solidifying their power and stifling competition due to the immense value and scale of their infrastructure.

GPUs, or graphics processing units, were initially developed for rendering graphics in video games but have since become integral to the artificial intelligence race. These chips are expensive, scarce, and primarily supplied by Nvidia Corp., a company whose market value exceeded $1 trillion last month due to the skyrocketing demand for GPUs. Developers typically purchase access to cloud servers from industry leaders like Microsoft Corp. and Amazon.com Inc., which heavily rely on GPUs to power their servers.

The adage “during a gold rush, sell shovels” holds true in the current AI infrastructure landscape. It is unsurprising that providers of AI infrastructure are capitalizing on the boom. However, there is a significant difference between the present situation and the mid-19th century California Gold Rush, where upstarts like Levi Strauss and Samuel Brennan made their fortunes. In the AI industry, most of the profits generated from selling AI services will flow into the coffers of tech behemoths such as Microsoft, Amazon, and Nvidia, companies that have long dominated the tech sector.

One contributing factor to this power consolidation is that while the costs of cloud services and chips are on the rise, the price of accessing AI models is decreasing. OpenAI, for instance, has progressively reduced the cost of accessing its GPT-3 model, first by a third in September 2022, then by a factor of 10 six months later. In June, OpenAI further slashed the fee for its embeddings model, which facilitates large language models in processing contextual information, by a staggering 75%. Sam Altman predicts that the cost of intelligence is on a trajectory toward near-zero.

In contrast, the expense of building AI models is increasing, akin to the struggle of acquiring toilet paper during the Covid-19 pandemic. Nvidia’s A100 and H100 chips set the gold standard for machine-learning computations, but the price of H100s has risen to over $40,000 from under $35,000 in just a few months. Furthermore, a global shortage hampers Nvidia’s ability to produce chips at a sufficient pace. Many AI startups find themselves waiting in line behind major customers like Microsoft and Oracle to secure these much-needed microprocessors.

More from aidigitalx

Day 3 of Shipmas: OpenAI Launches Sora, Its Text-to-Video AI Tool

OpenAI Sora AI video generator allows text-based video creation and editing, now live for ChatGPT Plus and Pro users globally.

OpenAI Kicks Off 12 Days of “Ship-mas” With Big Announcements, Day 4 Covered.

Why Mira Murati Quit OpenAI

Character.AI-Google Deal Raises Eyebrows Amid Regulatory Scrutiny

Microsoft’s new generative AI model for top-secret work with US intelligence agencies

Even OpenAI, despite its prominence, is reportedly awaiting H100 chips until spring 2024, according to a startup founder familiar with Nvidia. While OpenAI declined to confirm this information, Altman himself has voiced his struggles in procuring chips.

Big Tech companies possess a significant advantage over upstarts like OpenAI due to their direct access to critical GPUs and established customer bases. When Sam Altman exchanged 49% of OpenAI for Microsoft’s $1 billion investment in 2022, the amount of equity surrendered appeared substantial. However, this move can be seen as a strategic decision to align with a major cloud vendor, securing the future of AI companies.

Microsoft is reaping the benefits of this partnership as its chief financial officer, Amy Hood, announced that AI-powered services, including those powered by OpenAI, will contribute at least $10 billion to the company’s revenue. Hood labeled it “the fastest-growing $10 billion business in our history.” Microsoft’s offering, called Azure OpenAI, is pricier than OpenAI’s own service but provides enterprise-friendly features, satisfying security and compliance requirements for companies like CarMax and Nota.

Meanwhile, AI model creators constantly face talent migrations within their companies, making it challenging to maintain secrecy and product differentiation. Additionally, their costs are never-ending; after expending funds on cloud credits to train their models, they must also operate those models for their customers—a process known as inference. AWS estimates that inference constitutes up to 90% of the total operational costs for AI models, with a significant portion going to cloud providers.

This landscape sets the stage for a two-tiered AI business system. Those at the top, with substantial financial resources and influential connections, are provided with computing credits worth hundreds of thousands of dollars from cloud vendors like Amazon and Microsoft. A select few have even secured partnerships with venture capital investor Nat Friedman, who spent an estimated $80 million on GPUs to create a bespoke cloud service called the Andromeda Cluster.

In contrast, smaller AI companies find themselves relegated to the long tail, lacking the connections and resources necessary to train their AI systems, regardless of the ingenuity of their algorithms.

There is a glimmer of hope for these smaller enterprises: Big Tech firms may eventually see their products and services commoditized, loosening their stranglehold on the AI market. As the chip shortage eventually eases, GPUs will become more accessible and affordable. Competition between cloud providers is also expected to intensify as they encroach on each other’s territories. For instance, Google is developing its own GPU alternative called a TPU, while Nvidia is bolstering its cloud business to compete with Microsoft.

Furthermore, advancements like LoRA and PEFT, which make the process of building AI models more efficient, will reduce the need for extensive data and computing power. Consequently, AI models are poised to become smaller, necessitating fewer GPUs and infrastructure. These factors indicate that Big Tech’s dominance may not persist indefinitely.