EMBRACE AGENTS

“Transformers was done in the context of a lot of, like, like, TPUs, transformers were all done to solve a specific product need to some extent. Right? Like, the team's thinking about how to make translation better. In the case of TPUs, how do you, pay speech rec works? We suddenly have to serve it to 2,000,000,000 people. We don't have enough chips for it.”

“I think the most important perma skill in this era is, curiosity, I think, because if the machine can answer all your questions, you better have good questions. And the only way to have good questions, I think, is to pursue the things you were actually excited about from an early age and throughout your entire life. And I reflect on this because the only reason I'm here and working on this stuff is because I thought it was neat when I got, you know, nerds sniped in, in the interview process.”

“I care a lot about long term retention, and I would put all my points there because, I'm I'm really proud of the retention stats we have. Huge. But, ultimately, the sign of durable value is whether or not people are coming back in three months because that means you're really solving their problems. And I think things like revenue, they follow from that.”

“You know, you've seen what we did with Moveworks, the agentic front door to the enterprise, Inveso, which is human and nonhuman identity, and now the security move with Armis on top of an unbelievable core. By the way, we integrated these businesses in twenty days. Twenty days. So a lot of companies don't have the engineering power to say they can do hard things quickly.”

“And then you have to have a position that you take that brings something unique and variable to the equation. I always tell people, if two people are in the same room at the same time with the same opinion, one of them is redundant.”

“If an agent can do it as good or better, that's an easily easy economic decision to make, which is why, you know, there'll be 2,200,000,000 of these agents entering the workforce in the next couple of years. So, you know, there's gonna be a lot more agents than there will be people.”

“We are constantly trying to take these long term projects, which when you first announce them, slightly marginally looks ridiculous. Okay. You know, like, we're in the earliest stages of thinking about data centers in space. But to your earlier discussion around constraint inspires creativity. But if you take a twenty year outlook, right, where are you going to put most of these data centers? Really hard problems to solve.”

“I'm coming here as we just shipped Gemma four. And, it's a really good open source model. The frontier to Gemma four is both huge and not so huge in terms of time. Like, of, Gemma four is based on Gemini three architecture. You know, it's a very weird thing. Right? You're talking about a set of ways which can fit on a USB stick.”

“I think your question earlier on, like, you know, I think you were asking in the context of way more robotics, like, companies. I do think companies which are that's one advantage startups are gonna have. More AI native teams. And and, you know, you can probably get at it through your interview processes, etcetera. Whereas for us, we would have, like, retraining, transformation etcetera. And I think that that's maybe an advantage, like, the younger companies are gonna have.”

“Hey. The overton window shifted. We have like, I felt like the company was built for that moment. The vertical thing, it's it's it's not an accident or something. It was a very intentful we were on the seventh version of TPUs. So to me, we were behind in terms of frontier LLM models, but we had all the capabilities internally, and we had to execute to meet the moment.”

“So first off, Code Reds are a tool we use, to create focus. End of last year, we had one of those moments where we felt like we we need to show up for our users. We need to focus the things, focus on the basics, like reliability, performance, the way that talking to the model feels, making personalization really great. We just exited the code red, which we knew we would, with the launch of 5.3, which, you know, is a is a great model for the everyday user.”

“The thing that's already come first is the, domain specific agents. If you look at what's happening in in code, we're we're we're fully there. You know, it it's mind bending, but we've got so many engineers, who who don't open their IDE, like, ever. I won't be surprised if you see this happen for other forms of sort of quantitative knowledge work just because it happens to have the properties that code has. It's testable. You know if it worked or not. It's, you know, very RL friendly.”

“One other shift that I've noticed as our cloud agents have really taken off internally has been a shift from primarily individually driven development to almost this collaborative nature of development. For us, Slack is actually almost like a development an IDE, basically. We will have these issue channels or just, like, this product discussion channels where people are always at cursoring, and that kicks off a cloud agent. Oftentimes, I will kick off an investigation, and then sometimes I even ask it to get blamed and then tag people who should be brought in because it can tag people in Slack.”

“Waymo was a great example where I think we increased our investment two to three years ago when the rest of the world got pessimistic on it. When others, some of the people are backing off. For example, if Waymo had reached this point earlier, I think I would have invested the capital earlier. But I would have been glad to invest more capital in Waymo earlier, but we weren't at the level of maturity needed to do that.”

“Memory is definitely one of the most critical components now. There is no way that the leading memory companies are going to dramatically improve their capacity. So you have those constraints in the short term, but they they get more relaxed as you go out. By the way, I think it'll push a lot of innovations on. We will make these things 30 x more efficient.”

“For example, you know, 90% of our customer service cases now are managed by agents. And, you know, that that means, like, only 10% are actually involving people. And so there's a lifting and a shifting and a changing of the guard in terms of what people do for the critical thinking and the judgment calls that they have to make instead of the tactical, you know, work of just grinding out details.”

“But to give an example, like, search, you know, I was speaking with the teams. Right? Like, they now have for sub teams, like, latency budgets, like, in the milliseconds. You'll get 50% credit if so if you ship something which, you know, shaves off three milliseconds, you earn one point five milliseconds for your latency budget, and one point five milliseconds gets passed on to the user.”

“We think that over the coming months, the big unlock is not going to be one person with a model getting more done, like the water flowing faster, and we'll be making the pipe much wider. And so paralyzing more, whether that's swarms of agents or parallel agents, both of those are things that contribute to getting much more done in the same amount of time, but any one of those tasks doesn't necessarily need to get done that quickly.”

“Internally, we call it grind mode. There's a specific you have to start out by aligning, and there's, like, a planning stage where it will work with you, and it will not get, like, start grind execution mode until it's decided that the plan is amenable to both of you. We found that it's really important where people would give, like, very underspecified prompt and then expect it to come back with magic. If it's gonna go off and work for three minutes, that's one thing. When it's gonna go off and work for three days, probably should spend, like, a few hours upfront making sure that you have communicated what you actually want.”

“But GPUs are zero sum. And if you don't have more GPUs, you really have to figure out how do you make very, very hard trades and hate making hard trades for our users. Hence the desire to, have more GPUs, but, it's it's useful to start with the most zero sum trade off when you do your planning. So I think starting working backwards from GPUs, is usually a pretty pretty good idea.”

“Pillar two is the model coming back with a video of what it did. We have found that in this new world where agents can end to end write much more code, reviewing the code is one of these new bottlenecks that crop up. And so reviewing a video is not a substitute for reviewing code, but it is an entry point that is much, much easier to start with than glancing at some giant diff.”

“I bought the business for 5,500 notes, 7,000 with interest. If I make the payments, I keep it. If I miss a payment, they take everything away from me. What was really interesting about the story is I didn't have any money. But what I did have, because I worked there, is relationships with all the suppliers. So I got them to give me the first order on consignment. I said, I will always pay you back. This is not a favor. This is a chance for you to get your shelves filled up with your stuff that I'm gonna sell for you.”

“Phase one, tab autocomplete. People paid, like, $20 bucks a month, and that was great. Phase two, where you were iterating with these local models today, people pay, like, hundreds of dollars a month. I think as we think about these highly parallel kind of agents running off for long times in their own VM system, we are already at that point where people will be spending thousands of dollars a month per human, and I think potentially tens of thousands beyond.”

“Let's take that cost, and then let's take the cost associated with the human capital doing that instead of something else because the platform was doing the work for you. And then let's add up the cost of the GPU factory, the business model associated with the language model company, and the tokens that will materially affect their business model. We've actually done the math on this. And so for a simple application on our platform, it would be 10 times greater in cost to try to replicate it with a language model.”

“You know, this is the moment when leaders really matter. Because as the waters get choppy, you know, we see who's tough and who's not. And when the tide goes out, you wanna be fully dressed. And even if things don't look so good, you'll figure it out if you're fully dressed and you're ready for the battle.”

“Self awareness broadly has been a really big thing. The agent should understand how its environment works. It should understand how secrets work. Like, it needs to be self aware about its own harness and its environment. This is, like, one of the first things I learned at Dot when we launched was that I we had made the product work very well at a certain number of things, but didn't have complete self awareness of, like, its own boundaries. So people would be like, hey. Can you do this thing? And the thing was there and could be done. And the the product would be like, oh, no. Users will often attribute increased intelligence to a system that is more highly self aware.”

“I have a prediction for you. I predict that by the end of the year, I think it will take longer than people think and longer than we think for cloud and agents working in their own boxes to surpass local agents, but I think that crossover will happen before the end of the year. And probably by the end of the year, agents running in the cloud will be a multi like, more than two x the volume of local agents.”

“Pricings there's no world in which pricing doesn't significantly evolve when the technology is changing this quickly. Yeah. it's possible that, you know, in in the current era, having unlimited plan is like having unlimited electricity plan. You know? It just doesn't make sense because, like, you know, people may need a lot a lot of electricity, and they're getting a lot of value out of that. There's a reason you can't buy that.”

“We've got about 10% of the world coming to us now, 90% left to go. Right? There's so much more opportunity to to to to reach more people and introduce them to the way that AI can can can benefit fit them. But we're also really excited to go deeper. And that means taking the same billion users that find value in ChatGPT today, and actually providing more meaningful value in their world.”

“The big new thing here is that the agent will test its changes. So you can see here it worked for half an hour. That is because it not only took time to write the tokens of code, it also took time to test them end to end. So it started dev servers, iterate when needed. One of the other intuition pumps we use there is if a human gave you PR, asked you to review it, and you hadn't they hadn't tested it, you'd also be annoyed because you'd be like, only ask me for a review once it's actually ready.”

“ChatGPT originally was entirely free, and the reason for that was that it was intended to be a demo. And we're gonna wind it down after a month. We then realized that the demo went viral and people loved the demo, and it was actually a product. And but we realized it'd be a product. You can't take the product down every time you're at capacity. So we, you know, shipped subscriptions simply because it could shape the demand. It was a way of gracefully turning users away, and we had to turn away someone.”

“And there was an interesting learning that's relevant for these different model providers. It was something that would run a bunch of best of ends, but then synthesize and basically run like a synthesizer layer of models. And what we found was that at the time at least, there were strengths to using models from different model providers as the base level of this process. Like, basically, you could get almost like a synergistic output that was better than having a very unified, like, bottom model tier.”

“Now the other thing about that is if I give you the language model and it makes a mistake, you call me up and you say, hey, Bill. The language model made a mistake. And I say to you, the language model works. And you say, yeah. It works, but it made a mistake. Well, it's probably right, but it's not deterministic. And what we're learning too is people that run businesses understand that people make mistakes. They never will forgive software for making a mistake.”

“Right now, I think it's just tell me what I need to know. It's not like, let me come in and discover your problems. Well, let me bring you a solution because I have one in my pocket, and I can't wait to share it with you. It's like, you know my business. If you don't know my business, there's no conversation. But let's assume you know the business. Be very prescriptive.”

“Here's another example that we found really cool, which is we've actually turned since into a slash command as well, slash repro, where for bugs in particular, the model having full access to the to its own VM, it can first reproduce the bug, Make a video of the bug reproducing, fix the bug, make a video of the bug being fixed. And that has been the single category that has gone from, like, these types of bugs, really hard to reproduce and takes you tons of time locally, to when this happens, you'll merge it in ninety seconds or something like that.”

“I think Mark and I were, giving a demo of reasoning, in front of the, whole company. We're having to do a puzzle in front of everyone. And, I think one of the moments that made me totally feel the EGI is, like, we were in the middle of the demo, and everyone started laughing. I was like, wait. What what is funny? And then I stared at the screen because we're showing this chain of thought as it was streaming out of the model. And the model swore and said, like, oh, damn it. May I have to adjust because I realized I had made a mistake in the puzzle.”

“But now it is really acutely constrained. Right? So you spend a lot more time. I at least spend a dedicated hour a week thinking about that question at a pretty granular level. So I will know by projects and by teams the compute units they are using. Right? And, you know, or or at least I have that information, and I'm looking at it and assessing it.”

“At the end of an interview, he said to me, Bill, it's a very interesting interview. You're interesting guy. The HR department's gonna get in touch with you in the next couple of weeks. And I said to him, you know, mister Fuller, I don't think you completely understand the situation, sir. And he looks at me with a tilted head, like, what's this kid up to? And I said, I haven't broken a promise to my father in twenty one years, and I guaranteed I won't come home tonight with my employee badge in my pocket. And he goes, you know, Bill McDermott, as long as you haven't committed any crimes, you're hired.”