MONITOR AI SAFETY

“We think of it as trying to build advanced science that can say, when are we getting to the point that AI systems could improve themselves or speed up the pace of AI development? When will AI research feed on itself? The core capability for that might be software engineering and machine learning research ability.”

“In this case, we're talking about for a bus 4.6, something like tasks that take humans 12 hours to do, we predict that it will succeed at those tasks around 50 percent of the time. It turns out that when you plot using this particular difficulty measure, how performant AIs are relative to how long it takes humans to complete these tasks, we see an exponential increase in capabilities for AIs.”

“Sebastian, I think you should build a self diving car that can drive anywhere in the world. And my immediate reaction was, no, taking the technology we build for this empty desert and put it in the middle of Market Street in San Francisco is going to kill somebody. And Larry would come back the next day with the same idea, and I would give them the same answer, and both of us got increasingly more frustrated. God damn it, it can't be done, and eventually came and said, look, Sebastian, OK, care, I get it. You can't do it. I want to explain to Erk Schmidt the CEO at the time and Sergey Britt my cofounder, why it can't be done?”

“The second big reason that these cases are really important is that they appear to have opened up a crack in Section 230 of our Communications Decency Act here, which for 30 years has been essentially the foundation that the entire internet rests on. Section 230 is the law that says that in most cases, these platforms cannot be held liable for what their users post.”

“This is not about, 'oh, I got harmed by this particular piece of content.' This is about the design of the whole platform. The design feels defective. And the really crazy thing about these cases, Kevin, is that juries agreed with these plaintiffs for the first time. And they said, we like this theory. We think these products are defective.”

“In LA, a jury found that Meta and YouTube had been negligent in the way that they designed features, that they said were harmful to this plaintiff. They have to pay $6 million combined to this plaintiff. And then in New Mexico, the jury said, we believe that Meta has violated the state's Unfair Practices Act and has misled consumers about the safety of its products and has endangered children.”

“Safety mandates are moving toward hard compute caps.”

“The launch of Meta's Gemini 4 is a pivotal moment for the industry.”

“The open-weights ecosystem is evolving faster than many expected.”

“The tasks that come up in the wild are more likely to be messy in some sense. They involve working with other people. They involve working in much larger code bases or more open-ended problems, maybe with something even adversarial going on. We do tend to see that the AIs are less capable of working on these more messy problems.”

“And what that ends up meaning is that you keep on having these doublings of capabilities every, let's say, four months, it seems, on recent trends, where the next model is not merely going to have necessarily an hour longer time horizon, but perhaps be having some multiple of the time horizon of the previous model that's come out.”

“The main difference in their approach is how quickly they want to move. Anthony is very okay with risk. He gets one of these cars and he's driving it back, and he lives in Berkeley, works in Palauto. He's just using this car like the Bay Bridge every day, probably outside the bounds of what the team actually wanted, and he's not necessarily logging data. He's just enjoying his self driving car and taking it all over the place.”

“The human brain is a mobile processor. It weighs a few pounds. It consumes, I think, around 20 watts. If you compare that to what we see in a data center, instead of 20 watts, you could have 200 megawatts. Instead of a few pounds, you could have several million pounds. Instead of 100 hertz on the channel, you can have 10 billion hertz on the channel. Instead of electrochemical wave propagation at 30 meters per second, you can be at the speed of light, 300,000 kilometers per second. In terms of energy consumption, space, bandwidth on the channel, speed of signal propagation, you've got six, seven, maybe eight orders of magnitude in all four dimensions simultaneously.”

“OpenAI's latest policy suggestions focus heavily on international cooperation.”

“I think clearly the central reason is that we are bottlenecked on technical talent, on incredibly capable people to come work on these questions. I was on a METR work retreat recently where we were brainstorming 20, 30 of these, what seemed like world important problems, problems that we think no one else is going to get to if we do not get to them.”

“It shows you can trust the industry and market forces in coordination with the government. They were talking to the government about this, but they're not relying on some top down regulation in order to do this. They laid out a blueprint that seems to me very pragmatic that now that we're at this threshold, we're gonna sandbox these things.”

“One extraordinary fact from my perspective... is something like the R&D spend on compute of these companies has risen exponentially, of course, and in fact, it's risen exponentially at essentially the same rate as time horizon progress. You know, I think there's nothing necessary about that. You know, it doesn't mean by itself that if compute progress slows, then capabilities progress will also slow.”

“Yes. 2028 and you can see that on my blog from 2009.”

“In some ways, I actually think many people in the general public are ahead of the experts, because I think there's a human tendency. If I talk to non-tech people about current AI systems, some of the people say to me, oh, well, doesn't it already have like human intelligence? It speaks more languages than me. It can do math and physics problems better than I could ever do at high school. It knows more recipes than me. I was confused about my tax return and explain something to me or whatever. In what way is it not intelligent? But often people who are experts in a particular domain, they really like to feel that their thing is very deep and special and this AI is not really going to touch them.”

“Four point eight million Americans drive for a living. It's one of the most common jobs we have, and these workers do not plan to surrender to the California tech companies. They're doing this because they stand to make an unfathomable amount of money if they eliminate driving jobs for working class of people. These drivers are represented by unions backed by politicians and in cities across America blue cities. They're organizing. So far they're winning.”

“If it passes that, I would propose we then go into a second phase, which is more adversarial. And we say, okay, it passed the battery of tests, so it's not failing at anything in our standard collection of however many thousands of tests or whatever we have. Now, let's do an adversarial test. Get a team of people, give them a month or two or whatever. They're allowed to look inside the AI, they're allowed to do whatever they like. Their job is to find something that we believe people can typically do, and it's cognitive, where the AI fails at. If they can find it, it fails by definition.”

“The company realized it would wreak havoc. They ran their own vulnerability testing. They saw that it would allow offensive hacking and people to expose browsers and browser history, expose credit cards, you know, on the Internet. So, you know, what I like about this is they didn't need government to hold their hand on this.”

“I gave a talk to the Russell Group Vice Chancellor. So in the UK, the Russell Group is atop universities. I said to them, look, this AGI thing is coming, and it's not that far away. In 10 years, we're going to have it. And it's going to start being able to do a significant fraction of all kinds of cognitive labor and work and things that people do, right? We actually need people in all these different aspects of society and how society works to think about what that means in their particular area. So we really need every faculty and every department that you have in your university to take this seriously and think, what does it mean for education? What does it mean for law? What does it mean for engineering?”

“These are models with massive step function improvements and intelligence, and they're just too smart to be released immediately. You know, and by the way, there was nothing that said that every time you finish a model you gotta immediately release it GA. So they set up this idea of sandboxing, building defensive alliances, in order to move away from that regime.”

“Sebastian Mallaby, the author of “The Infinity Machine,” joins us to talk about the three years he spent with Demis Hassabis and those closest to Google DeepMind. We discuss his new book on Google DeepMind and Demis Hassabis' quest to build super intelligence.”

“I saw that all the teams treated this like a hardware problem. They looked at this and say, we have to build a bigger wheels and bigger chassis and so on. And I looked at this and said, about wait a minute. The challenge really is to build a self driving car. They can drive for the desert. I can get a rental car. They can do it just fine, provided as a person insight and the challenges we need to take the person out of the driver's seat and replace it by computer. That is not a problem with bigger tires. That's actually be a software problem.”

“I had experiences of losing people in my life to traffic accidents, and I felt we lost over the million people in the world to traffic accidents. Wouldn't it be amazing if Dabok invented something that would save a million lives a year.”

“In the last moments of Alane Herzburg's life, the robot spent an indefensible five point six seconds trying and failing to guess the shape in the road there was a human body pushing a bike. Over those five point six seconds, the robot kept reclassifying our whishing an unknown object a vehicle a bicycle. During that time, spent wondering the car did not slow down. Soon after Elaine Hertzberg's death, Uber halted its testing program.”

“Weimos says, and I think this is correct, that it's roughly eighty brass safer in terms of crashes are severe enough to turn down an airbag. Crashes severe enough to cause an injury, and also crashes involving vulnerable road users like pedestrians or bicyclists. So far it's been better than human drivers, and so far, I think the case for allowing them they continue. The experiment is very strong.”

“So my definition of AGI, or sometimes I call minimal AGI, is an artificial agent that can at least do the kinds of cognitive things people can typically do. And I like that bar because if it's less than that, it feels like, well, it's failing to do cognitive things that we'd expect people to be able to do. So it feels like we're not really there yet. We're not there yet, and it could be one year, it could be five years, I'm guessing probably about two or so.”

“Tony Tether, who had been a door to door salesman in his use definitely has that flare in that way of thinking, says, let's have a contest. Let's see who can put all of these ingredients that we've developed together into a proper self driving car. His original idea is we'll drive him down the Las Vegas Strip that's almost immediately next because it's insane.”

“So even if the AI does develop quite quickly, in its purely cognitive sense, I don't think robotics will be at the point at which it could be a plumber. And then even when that is possible, I think it's going to take quite a while before it's price competitive with a human plumber, right? And so I think there are all kinds of work which is not purely cognitive that will be relatively protected from some of the stuff. The interesting thing is that a lot of work which currently commands very high compensation is sort of elite cognitive work. It's people doing, I don't know, sort of high-powered lawyers that are doing complex merger and acquisition deals across the globe and people doing advanced stuff in finance.”

“Let's spend a hundred days using advanced AI to find and to fix and to harden these software vulnerabilities before hackers exploit them. Now what I think this represents, Jason, is a threshold that we're crossing. Mythos and Spud, which is going to be out from OpenAI any day now, represent the beginning of what I would call AGI models.”

“These are what are called bellwether cases. These are the cases that if successful are going to open the floodgates for lots of other people to sue under the same theory. It seems like juries are just going to be really, really sympathetic to these claims.”

“You might say, for example, I don't know, lying is bad, right? So we're not going to lie. But you could be in a particular situation where, I don't know, you know, there's some bad people coming to get somebody. And if you tell a lie, you can save their life. And then the ethical thing to do is maybe to lie. And so the simple rule is not always adequate to really make the right decision. Sometimes you need a little bit of logic and reasoning to really think through.”

“METR is a research nonprofit based in the Bay Area... dedicated to advancing the science of measuring whether and when AI systems might pose catastrophic risks to humanity as a whole, focused specifically on threats that come from AI autonomy or AI systems themselves. We think it sets the stakes for conversations about AI misalignment.”

“We're seeing unprecedented multimodal reasoning in this new release.”

“I think Anthropic has proven that it's very good at two things. One is product releases. The second is scaring people. And we've seen a pattern in their previous releases of, at the same time, they roll out a new model or new model card, something like that. They also roll out some study showing really the worst possible implication of where the technology could lead.”