PRIORITIZE INNOVATION

“I would say about 90% of the breakthroughs that underpin the modern AI industry were done either by Google Brain or Google Research or DeepMind. Those labs that have capability to invent new algorithmic ideas are going to start having bigger advantage over the next few years. The last set of ideas are sort of all the juices being rung out of them.”

“I've got a probability distribution around the timings, but I would say there's a very good chance of it being within the next five years. So that's not long at all. We used to do this extrapolation of compute and algorithmic progress, and basically we predicted around 20 years it would take from when we started out, and I think we're pretty much on track.”

“I don't think we have hit the limits. I think it's a bit more nuanced than that. So of course when the leading companies all started building these large language models, you're getting enormous jumps with each generation of new system. At some point that had to slow down, so it's not continuing to be exponential. But that doesn't mean there isn't great returns still for scaling the existing systems up further.”

“I think compute is the big one, not just for the obvious reason of scaling up your ideas and your systems as the scaling laws as they're called, but the other thing you need a lot of compute for is for doing experiments. The computers, the cloud is our workbench basically. So if you have a new algorithmic idea, you've got to test it at a reasonable scale, otherwise it won't hold when you put it into the main system.”

“I sometimes quantify the coming of AGI is like 10 times the industrial revolution at 10 times the speed. We've been very consistent how we define AGI as basically a system that exhibits all the cognitive capabilities the human mind has. That's important because the brain is the only existence proof we have that we know of in the universe that general intelligence is possible.”