Raphael Thys
Baseten's cofounders are (from left) Amir Haghighat, Tuhin Srivastava, Phil Howes and Pankaj Gupta.
Companies have spent billions on computing costs to train artificial intelligence models, but the costs are starting to shift to when the AI is actually used — say, the compute ChatGPT uses to answer a question in real time. That process, called “inference,” accounted for 40% of Nvidia’s computing revenue last year, the company shared in its latest earnings report. “If Nvidia’s business is 90% training and 10% inference, you could argue that AI is still in research,” CEO Jensen Huang said in a recent Wired interview. Having so much computing power dedicated to inference, he said, shows that “AI is finally making it.”
Become a member and unlock exclusive access to diverse, in-depth journalism not available anywhere else for less than $1/week.
- Premium access to exclusive events, thought-provoking conversations with global leaders and more, all available on-demand.
- Elevated browsing experience with fewer ads and unlimited article saving power an enhanced reading experience.