BUILD FROM SCRATCH

“Anthropic is known for betting very hard on code, which is called code thing, is working out for them right now. So I think that even if the ideas flow pretty freely, so much of this is bottlenecked by human effort and kind of culture of organizations where anthropic seems to at least be presenting as the least chaotic, it's it's a bit of an advantage.”

“It's held for 13 orders of magnitude of computer something. Like, why would it ever end? So I think fundamentally it is pretty unlikely to stop. It's just like, eventually, we're not even gonna be able to test the bigger scales because of all the problems that come with more compute.”

“DeepSeek is kind of losing its crown as the preeminent open model maker in China and the likes of, Z dot AI with their GLM models, MiniMax's models, Kimmy Moonshot, especially in the last few months, have shown more brightly. The new deep seek models are still very strong, but that's kind of a it could look back as a big narrative point where twenty twenty five deep seat came and then all and it kind of provided this platform for way more Chinese companies that are releasing these fantastic models.”

“The next few years are definitely gonna be an increased value on physical goods and events and then even more pressure on slop. So there'll be so they'll keep the slop is only starting. The next few years will be more and more diverse versions of slop. Hoping that we society, drowns in slop enough to snap out of it and be like, we can't. Like, none. Like, it just doesn't matter. We all can't deal with it. And then, like, the physical has such a higher premium on it.”

“Llama was the, I would say, pioneering open weight model, and then Lama one, two, three, a lot of love. But I think then I think what happened just hypothesizing or speculating. I think the, leaders at Meta, like, the upper, executives, they I think they got really excited about Llama because they saw how popular it was in the community. And then I think the problem was trying to, let's say, monetize the open not monetize the open source, but, like, kind of use the open source to make a bigger splash.”

“Building an element from scratch is a lot of fun. It's also a lot of to learn. And like you said, it's probably the best way to learn how something really works because you can look at figures, but figures can have mistakes. You can look at con concepts, explanations, but you might misunderstand them. But if you see the there is code and the code works, you know it's correct.”

“I don't think nowadays, 2026, that there will be any company who is, let's say, having access to a technology that no other company has access to. And that is mainly because researchers are frequently changing jobs, changing labs, they, rotate. So I don't think there will be a clear winner in terms of technology access. However, I do think there will be, the differentiating factor will be budget and hardware constraints.”

“I was training the Gwent three base model with RLVR on math 500. The base model had an accuracy of about 15%. Just 50 steps, like in a few minutes with RLVR, the model went from 15% to 50% accuracy. And the model you can't tell me it's learning anything about fundamentally about math in 50 steps. So the knowledge is already there in the pre training. You're just unlocking it.”

“Anthropic lost in court and was owed $1,500,000,000 to authors. Anthropic, I think, bought thousands of books and scanned them and was cleared legally for that because they bought the books, and that is kind of going through the system. And then the other side, they also torrented some books. And I think this torrenting was the path where the court said that they were then culpable to pay this billions of dollars to authors, which is just, like, such a mind boggling lawsuit that kinda just came and went.”

“I disagree with some of their presumptions and dynamics on how it would play out, but but I think they did a good they did good work in the scenario defining milestones. The camp that I've fallen to is that, like, AI is, like, so called jagged, which will be excellent at some things and really bad at some things. So I think that when they're close to this automated software engineer, what it will be good at is that traditional ML systems and front end, the model is excellent at. But the distributed ML, the models are actually really quite bad at.”