Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute

“And so you sort of have this, like, dynamic that is quite interesting in that. An h one hundred is worth more today than it was three years ago.”

“Dario, when he was on your podcast, was very, very, like, conservative. He's like, you know, I'm not gonna go crazy on compute because if my revenue inflects at a different rate, at a different point, I don't wanna go bankrupt. But in reality, you know, he's definitely missed the pooch in terms of, like, going, like, OpenAI, which was, let's just sign these crazy fucking deals. And OpenAI has kind of got way more access to compute than Anthropic by the end of the year. And so what does Anthropic have to do to get the compute? Well, they have to go to lower quality providers that they would not have gone to before.”

“Google sold, I think, a million was it the v sevens, the Ironwoods, to Anthropic? DeepMind people are like, this is insane. Why did we do this? But then Google Cloud people and Google executives saw a different, like, thought process. They saw this dislocation. They negotiated a deal, and they were able to get access to these to this compute before Google realized. We saw capacity on anthropic or sorry, on TPUs go up by a significant amount over the course of those six weeks. Google even had to go to TSMC and explain to them why they needed this increase in capacity because it was so sudden. But that a lot of that capacity increase was for selling to Anthropic. Because Anthropic saw it before Google.”

“ASML makes the world's most complicated machine, I e, an EUV tool, and the selling price for those is $304,100,000,000 dollars. And currently, they can make about 70. Next year, they'll get to 80. Even under very aggressive supply chain expansion, they only get to a little bit over a 100 by the end of the decade. ASML's commented their supply chain has over 10,000 people in it.”

“I believe an iPhone has 12 gigabytes of memory. Each gig cost used to cost roughly 3 or $4. That's $50. But now the price of memory is, like, tripled. Let's call if it's now, it's $12 per gig for DDR. So now you're talking about a $150 versus $50. A $100 increase in cost on Apple. Apple has some margin. They're not just gonna eat the margin. So now that's a $100 cost increase. The NAND also has the same sort of, like, market. So in fact, you know, it's probably a $150 increase on the on the iPhone. That means the end consumer is paying $250 more for an iPhone.”

“Used to be 1,400,000 smartphones were sold a year. Now we're at, like, 1.1. But our projections are we maybe get down to, like, 800,000,000 this year. And next year, like, 600 or 500,000,000. They see Xiaomi and Oppo are cutting low end and mid range smartphone volumes by half. Today, you already see all the memes, like, on PC subreddits and PC gaming PC Twitter is, like, cat dancing videos. And it's like, this is why memory prices has doubled, and you can't get a new gaming GPU.”

“Space data centers effectively are not limited by, you know, hey. We have this energy advantage. It's actually just limited by the same contended resource. We can only make 200 gigawatts of chips a year by the end of the decade. So what are we gonna do to get that capacity? It doesn't matter if it's on land or in space. Elon doesn't win by doing, you know, 20% gains. Elon never wins that way. Elon wins when he swings for the fences and does 10 x gains.”

“In isolation, you almost always wanna go with a smaller model that gets RL'd faster and gets deployed into research and development. So you can build the next thing and get more compute efficiency wins. And then this compounding effect of, oh, I made a smaller model that I r l'd more that I then deployed into research and development earlier, and I spent less compute on the training itself because I was able to allocate more compute to the research. This compounding effect of being able to do the research faster and faster is potentially a faster takeoff.”

“Huawei is arguably the only company in the world that has all the legs. Huawei has cracked software engineers. Huawei has cracked networking technologies. That's, in fact, their biggest business historically. And they have cracked AI talent. But furthermore, beyond NVIDIA, they actually have better AI researchers. And furthermore, beyond NVIDIA, they have their own fabs. It's very arguable that Huawei, if they had TSMC, would be better than Nvidia.”

“If you ship out all the process engineers and assuming it's, like, hot enough that you destroy the fabs, no one has all the fabs in Taiwan now, which is a big risk. These tools actually use a lot of semiconductors, which are manufactured in Taiwan. So it's like a snake eating its own tail sort of like meme because you can't make the tools without the chips from Taiwan, which you can't use without the tools in Taiwan. Just shipping out all the engineers and blowing up the fabs means China has a stronger semiconductor supply chain than the rest of the world.”