WATCH RETENTION

“One of the first big differences is the eating out culture in China is very, very high and very, very affordable. Eating out in China is about as affordable as cooking at home. And as a result, nobody cooks. And there are a lot of reasons for this, but we can go super deep on retail history and grocery history in China. But anyways, but that's a huge phenomenon. I think second, because of labor market dynamics, both the availability as well as the cost of labor in the Chinese market, that's allowed a lot of these businesses to make this activity almost as affordable, as if you were to pick up the orders yourself.”

“One of the first big differences is the eating out culture in China is very, very high and very, very affordable. Eating out in China is about as affordable as cooking at home. And as a result, nobody cooks. And there are a lot of reasons for this, but we can go super deep on retail history and grocery history in China. But anyways, but that's a huge phenomenon. I think second, because of labor market dynamics, both the availability as well as the cost of labor in the Chinese market, that's allowed a lot of these businesses to make this activity almost as affordable, as if you were to pick up the orders yourself.”

“When our backs were against the wall and we had, I think, less than two weeks of cash runway. And we had a terrible night where every single order was late. This was a Stanford football game in 2013 and I had trouble raising the seed round. We made the decision to refund everyone. That cost us over 40% of the bank accounts. So when you have two weeks of runway, 40% of the bank accounts. And we baked everyone cookies and deliver them at 5 a.m. before everybody woke up.”

“It just turns out it's extraordinarily difficult, however, unless you're a large brand or a house of brands, someone like DoorDash, to be able to attract enough customers to make that math work. ... For businesses like Chipotle, who tend to identify real estate in fairly expensive areas, there's high opportunity costs of what you do with that space. Yes, you're right. One choice is to turn it into a delivery-only kitchen, and sometimes that happens. But there's also a massive opportunity to recoup the expense.”

“It just turns out it's extraordinarily difficult, however, unless you're a large brand or a house of brands, someone like DoorDash, to be able to attract enough customers to make that math work. ... For businesses like Chipotle, who tend to identify real estate in fairly expensive areas, there's high opportunity costs of what you do with that space. Yes, you're right. One choice is to turn it into a delivery-only kitchen, and sometimes that happens. But there's also a massive opportunity to recoup the expense.”

“From the product perspective, yes, because at the end of the day, any consumer product is judged very simply by its retention and its usage. That's how you know whether you have a differentiated product. I think it's very easy to have differences in opinion about whose app do you like more or whether or not certain apps look similar or different. At the end of the day, though, if our app is performing at a higher retention, much higher retention and frequency of use than others, that's how we know whether or not the things that we say actually are making a difference to customers.”

“Drones obviously can do a lot of these longer distance orders. And so we've been doing drone deliveries actually for a couple years now, mostly outside of the United States. Outside of the US. Places like Australia, we're going to bring them to Europe, bring them to the United States as well. But again, you have all of those problems you have to solve. The autonomy is a little bit easier. You still have a routing problem, you have hardware problems. Obviously, you still have permits and regulation, set up, loading inside different stores, things like that.”

“If you look at the country, a lot of this growth is happening in the south of the country. And that's been true for a couple of decades now. And they tend to be correlated, meaning if it's easy for me to build apartment units and to build just construction in general, it tends to be a bit easier to also get the licenses to open up a restaurant. That's good. There's some bright spots and it's a nationwide maybe negative trend when it comes to permitting.”

“Drones obviously can do a lot of these longer distance orders. And so we've been doing drone deliveries actually for a couple years now, mostly outside of the United States. Outside of the US. Places like Australia, we're going to bring them to Europe, bring them to the United States as well. But again, you have all of those problems you have to solve. The autonomy is a little bit easier. You still have a routing problem, you have hardware problems. Obviously, you still have permits and regulation, set up, loading inside different stores, things like that.”

“From the product perspective, yes, because at the end of the day, any consumer product is judged very simply by its retention and its usage. That's how you know whether you have a differentiated product. I think it's very easy to have differences in opinion about whose app do you like more or whether or not certain apps look similar or different. At the end of the day, though, if our app is performing at a higher retention, much higher retention and frequency of use than others, that's how we know whether or not the things that we say actually are making a difference to customers.”

“If you look at the country, a lot of this growth is happening in the south of the country. And that's been true for a couple of decades now. And they tend to be correlated, meaning if it's easy for me to build apartment units and to build just construction in general, it tends to be a bit easier to also get the licenses to open up a restaurant. That's good. There's some bright spots and it's a nationwide maybe negative trend when it comes to permitting.”

“I think when you have high volumes of activity, I think keeping the reliability as reliable as the electricity we have or the water inside of our buildings, that is extraordinarily difficult. ... One of the biggest things that we're going to have to do before we can just fulfill the items, which is what we'll get to, is where are the items and what are the items? There's tens of millions of items literally inside these cities, whether it's in the US or different countries within Europe, other parts of the world, they're not catalogued.”

“We've got about 10% of the world coming to us now, 90% left to go. Right? There's so much more opportunity to to to to reach more people and introduce them to the way that AI can can can benefit fit them. But we're also really excited to go deeper. And that means taking the same billion users that find value in ChatGPT today, and actually providing more meaningful value in their world.”

“We have our own mapping system, for example, that we built. Why do we build our own? Well, it's because we care much more than any third-party mapping system of exactly where the last two feet, forget 20 or 200 feet, of some apartment unit door is, for example. And we know if we reliably deliver to that door, and that we saw that the pin actually hit exactly where, we have a little bit more fidelity in whether something was dropped off. We build profiles of customers, I'm sure you do too, of their behavior and what they tend to say.”

“But GPUs are zero sum. And if you don't have more GPUs, you really have to figure out how do you make very, very hard trades and hate making hard trades for our users. Hence the desire to, have more GPUs, but, it's it's useful to start with the most zero sum trade off when you do your planning. So I think starting working backwards from GPUs, is usually a pretty pretty good idea.”

“Pricings there's no world in which pricing doesn't significantly evolve when the technology is changing this quickly. Yeah. it's possible that, you know, in in the current era, having unlimited plan is like having unlimited electricity plan. You know? It just doesn't make sense because, like, you know, people may need a lot a lot of electricity, and they're getting a lot of value out of that. There's a reason you can't buy that.”

“I think Mark and I were, giving a demo of reasoning, in front of the, whole company. We're having to do a puzzle in front of everyone. And, I think one of the moments that made me totally feel the EGI is, like, we were in the middle of the demo, and everyone started laughing. I was like, wait. What what is funny? And then I stared at the screen because we're showing this chain of thought as it was streaming out of the model. And the model swore and said, like, oh, damn it. May I have to adjust because I realized I had made a mistake in the puzzle.”

“I think the most important perma skill in this era is, curiosity, I think, because if the machine can answer all your questions, you better have good questions. And the only way to have good questions, I think, is to pursue the things you were actually excited about from an early age and throughout your entire life. And I reflect on this because the only reason I'm here and working on this stuff is because I thought it was neat when I got, you know, nerds sniped in, in the interview process.”

“ChatGPT originally was entirely free, and the reason for that was that it was intended to be a demo. And we're gonna wind it down after a month. We then realized that the demo went viral and people loved the demo, and it was actually a product. And but we realized it'd be a product. You can't take the product down every time you're at capacity. So we, you know, shipped subscriptions simply because it could shape the demand. It was a way of gracefully turning users away, and we had to turn away someone.”

“So first off, Code Reds are a tool we use, to create focus. End of last year, we had one of those moments where we felt like we we need to show up for our users. We need to focus the things, focus on the basics, like reliability, performance, the way that talking to the model feels, making personalization really great. We just exited the code red, which we knew we would, with the launch of 5.3, which, you know, is a is a great model for the everyday user.”

“The thing that's already come first is the, domain specific agents. If you look at what's happening in in code, we're we're we're fully there. You know, it it's mind bending, but we've got so many engineers, who who don't open their IDE, like, ever. I won't be surprised if you see this happen for other forms of sort of quantitative knowledge work just because it happens to have the properties that code has. It's testable. You know if it worked or not. It's, you know, very RL friendly.”

“When our backs were against the wall and we had, I think, less than two weeks of cash runway. And we had a terrible night where every single order was late. This was a Stanford football game in 2013 and I had trouble raising the seed round. We made the decision to refund everyone. That cost us over 40% of the bank accounts. So when you have two weeks of runway, 40% of the bank accounts. And we baked everyone cookies and deliver them at 5 a.m. before everybody woke up.”

“I think when you have high volumes of activity, I think keeping the reliability as reliable as the electricity we have or the water inside of our buildings, that is extraordinarily difficult. ... One of the biggest things that we're going to have to do before we can just fulfill the items, which is what we'll get to, is where are the items and what are the items? There's tens of millions of items literally inside these cities, whether it's in the US or different countries within Europe, other parts of the world, they're not catalogued.”

“I care a lot about long term retention, and I would put all my points there because, I'm I'm really proud of the retention stats we have. Huge. But, ultimately, the sign of durable value is whether or not people are coming back in three months because that means you're really solving their problems. And I think things like revenue, they follow from that.”

“One of the most difficult things is how do you actually staff your restaurant? This is the number one challenge, this has always been the number one challenge. And I think there's no easy ways around this really. And I think because the cost of labor only goes in one direction, only goes up, restaurants are increasingly making this choice of on the continuum of service to manufacturing, where do I want to sit on that spectrum? I think that is one very big trend, that more and more restaurants feel like they have to go towards the ends of the spectrum.”

“We have our own mapping system, for example, that we built. Why do we build our own? Well, it's because we care much more than any third-party mapping system of exactly where the last two feet, forget 20 or 200 feet, of some apartment unit door is, for example. And we know if we reliably deliver to that door, and that we saw that the pin actually hit exactly where, we have a little bit more fidelity in whether something was dropped off. We build profiles of customers, I'm sure you do too, of their behavior and what they tend to say.”

“One of the most difficult things is how do you actually staff your restaurant? This is the number one challenge, this has always been the number one challenge. And I think there's no easy ways around this really. And I think because the cost of labor only goes in one direction, only goes up, restaurants are increasingly making this choice of on the continuum of service to manufacturing, where do I want to sit on that spectrum? I think that is one very big trend, that more and more restaurants feel like they have to go towards the ends of the spectrum.”