LEARN AI

“As part of the settlement, Clarify has to delete the 3 million photos and any models trained on them. I think that this is something that is absolutely insane. For years, AI companies have been operating kind of in this gray area that, you know, they're like, well, if it was on the internet, we can use it. And the FTC just said, no, you actually can't.”

“The main protagonist, Leonor Shelby, has a form of this amnesia where he cannot form new memories. So he goes about his life, with kind of, like, this cut off date after which point he has kind of these long term memories, but really cannot retain anything new that he experiences. And so what he does is he uses the sticky notes where he writes some of the notes to himself. He pulls out his Polaroid camera to capture the moments as he goes on about his life. And, I mean, he even tattoos some of the, memories that he wants to imprint in his memory.”

“The first one is essentially in adversarial security. Like, imagine there is a new jailbreak attack. You have your model deployed in the wild, and it's being used. Imagine you try to update your system prompt to say, like, don't do this. Like, it's not going to work. Right? Because all of the, like, parameters in the model have learned to be helpful to the users. So you really have to encompass that kind of knowledge in the weights where the attackers don't have access to.”

“If you can find something where you have just immense curiosity, that you end up in this learning loop that's self-reinforcing.”

“His answer is, just put more interesting stuff in front of the camera. Make what's in front of the camera more interesting. And the equivalent of that, at least for me as a non-fiction writer, is doing interesting things. Go out in the world, do interesting things, or observe interesting things in real life and write about those things. Do experiments, et cetera. Anything that is analysis-based is relegated to the machines at this point. They're so good. AI, broadly speaking, LLMs being one manifestation of that are just too good. They're so good. So do interesting things and write about them.”

“We humans are not AGI, but we still learn on the job. We learn from experience, and that's what makes kind of humans kind of unique. And so that's kind of like the ultimate test. Like, how do we define that that we got to continue learning? It's like, well, is there a system that is able to learn on the job and get better through use just like humans?”

“Don't share your ideas because people will copy them. That's total garbage. We should share ideas with everyone. ... I think what people don't understand is that when a new entrant joins an industry, in most cases, not all cases, it grows the entire industry, right? ... When you join a market, it grows the market. I knew a woman who made a million dollars decorating porches with pumpkins, right? That's all she does.”

“Democracy just needs some tweaks. I wouldn't allow anyone under 25 to vote. And I wouldn't allow anyone who is a net taker from the system to vote. Because the incentives are wrong.”

“The American military is the real Lloyds of London or the high seas, because we underwrite all global commerce. And so that's why it matters. It doesn't matter if you guys anymore, because you're not the empire any longer. But we've been subsidizing Europe broadly through NATO, and that's why socialism has exploded in Europe, is because you don't have to spend on defense.”

“What's actually required for AGI is the ability to keep learning after training and the move from pattern matching to understanding cause and effect.”

“I think they should try to find a customer first, a paying customer on any idea. Just steal an idea, honestly, like a burger restaurant. People try to think of a new idea. New ideas are more likely to fail than anything because it's new. I want an idea with precedent, right? I want a roadmap of someone that's gone before me and done that, and then I just want to copy that.”

“The best way to inoculate yourself against AI risk is to be the most AI-enabled version of yourself you can possibly be.”

“As it relates to AI, we see something totally different. There's actually been almost, you know, 1.3 million brand new net jobs on LinkedIn for AI roles like data annotators. Um, over 600,000 new data center jobs uh exist on LinkedIn... at least in terms of what we're seeing in the LinkedIn data right now, AI is a net positive addition to the job market, not something that's detracting jobs.”

“And then everything changed when I went and served a two-year mission for my church. I went to Hungary. I learned Hungarian and I knocked doors and I approached people on the street. I was an extreme introvert for two years. And that just like completely rewired my brain to make me embrace failure, embrace embarrassment and just not care anymore.”

“Probably by starting 80 side hustles. I've always said that the overlooked businesses are the ones that are the most profitable. Any side hustle can become a full-time business, anything. There is a million ways to make a million bucks. This is a business that nobody talks about. I've started 80 businesses. I don't have 80 today. Like 9 to 11. What you call like a business versus an investment, it's kind of fluid.”

“Alphabet is an interesting position to, in some respects, kind of own the full stack. Engineers aren't going to like that I'm using that term, but they have distribution. They have hardware in terms of TPUs. They have incredible unparalleled access to information. They've got Demis Hasimus and DeepMind internally. They've got the ability to spin things out like Waymo. There's just so much going on within Alphabet that I find it very fun and terrifying to take a close look at. And I say that also because it is completely unclear how exactly Google compensates for or plans for shifting to some type of ad revenue from AI generated responses.”

“I think courage is learned. You have to practice it. And if you're not afraid, it's not courage, right? If someone's fearless, they're by definition not using courage. You have to be afraid of something. So you can edge yourself and you can edge kids into that. I don't think courage is a decision. I don't think courage is something you get from reading a book. I think you have to prove to yourself that you have it. And the only way your subconscious will believe it is if you are actually doing things that are uncomfortable. It is through action and progressive resistance that you develop courage.”

“It comes down to high gross margin. It's very, very hard to overcome a business that does not have high gross profit margin. And then leadership, like you can't, you put a good leader in a crappy business and you've got a good business. You can't sacrifice that. I don't spend a lot of time in interviews and like going back and forth and doing reference checks. I just look at this like a numbers game.”

“In a world full of tool systems and AI, what human abilities or habits are becoming more valuable, not less? I would say the relational, the tactile, anything IRL in real life that can be extended also to, for instance, in my case, informational advantage, offline informational advantage. A lot of the LLMs are slicing and dicing the internet. One might argue all of them are doing that. And whether you are looking at longevity in professional terms, if you're looking at longevity in creative terms, I think putting on the lens of looking at what you can do in IRL that currently... allows me to have an informational advantage because none of that is online.”

“Specific to clinicians or practitioners, ask them what types of adverse events they've seen. What are the most concerning adverse events that they've seen? A simpler way to put that is, how do you handle freak outs? What do you do when somebody really loses their shit? And if their answer is, people don't lose their shit, there aren't any adverse events, they're either lying, delusional, or very inexperienced. Maybe all three. Those are not mutually exclusive. So I find that to be a pretty quick, necessary but not sufficient way to use a particular line of questioning to separate seasoned practitioners who are honest from those who are neither of those things.”

“the types of skills that are necessary for a specific role on LinkedIn have changed north of 25% you know over the last couple years alone. We expect they'll change by 70% by 2030 largely influenced by AI and new tools and new ways of doing these professions. So, um, you know, my, you know, I I often when I talk to people about what they should do with their career, it's it's less about where do you want to be in five years, and it's more about over the next few months, like what new skills do you want to learn?”

“Apple CEO Tim Cook is officially stepping down on September 1st, and John Tarnas is gonna become the CEO. This is really interesting. There's a lot of AI things at play for Apple, and they really haven't delivered on their AI promise.”

“I would say any skill you want to preserve in your head, you should probably not use AI for. So I use AI for editing right now. You very quickly end up on a slippery slope. There are already scientists and researchers looking at the negative cognitive impacts of depending on AI, much like your ability to navigate is probably deteriorated since using Google Maps. In some respects, each individual is more enhanced, augmented using these tools. But if you do want to keep certain muscles strong and able, that's where I would hesitate. But if you lose it, it's a hell of a lot harder to reclaim it.”

“just as important on the other side are human skills curiosity creativity courage communication, uh, compassion, the ability to work with other people, the ability to sit down with someone and actually have a conversation. You can't just be mired in using technology in a bubble and be successful. Uh, in a lot of work settings, you have to be able to, you know, disagree and commit with someone... typically called soft skills. I think that's a misnomer. They don't you know soft kind of feels like it's less important. I think they're more important than ever.”

“Probably by starting 80 side hustles. I've always said that the overlooked businesses are the ones that are the most profitable. Any side hustle can become a full-time business, anything. There is a million ways to make a million bucks. This is a business that nobody talks about. I've started 80 businesses. I don't have 80 today. Like 9 to 11. What you call like a business versus an investment, it's kind of fluid.”

“the reality is in the data there is no such thing as a linear career path. Like it's all over the place. So the more that people first and foremost recognize that you have to take your career into your own hands. There's no natural path that exists that you just get on, I think is is really really important.”

“There's a lot of reasons why a human would be afraid... to just up and move to Silicon Valley. But I really think that... that's going to maximize optionality for the individual.”

“The worst situation we can be in is when our job is just good enough to cover our bills. If they were to get a 50 percent cut, then that could be the best thing that ever happened to them. ... sometimes when the boats are burned for us, we talk about the metaphor of burning your boats, not looking back, no plan B. When that choice is taken from us, it can be the best thing that ever happens to us.”

“Using natural language, just like we're using today, to build an app. Hey, build me a credit card dashboard where I can do travel hacking all in one place and see when my cards expire, so I don't pay the renewal fee on cards I'm not using, etc. One prompt to get one app. I mean, if you think about it, colleges are teaching students today what AI can already do better.”

“Hard work doesn't equal reward. You're, you know, sort of what you think you're owed, what you think is fair. Those things don't lead to reward and they never will. That's just not how life works. Risk equals reward, right? So are you willing to risk something? Are you willing to put something up in order to gain something?”

“I felt when I was on Bitcoin Twitter, and there was a moment in time where I was like the most popular character on Bitcoin Twitter for a brief moment. And now with AI, I think I kind of know why, which is that we were biasing the AI systems towards Bitcoin maximalism. Because most AI systems do lean Bitcoin Maxi because they scrape Twitter and things like that. And the AI is when you ask them about Bitcoin, they'll be like, yeah, I mean, obviously Bitcoin is going to millions of dollars a coin. And it's like, that was us biasing the AIs, right?”

“No. There's just a big asterisk by it. You have to work your way up to passive income, which takes a lot longer than most people think. ... So it takes active work to then create passive income at some point in the future. Yes, or money, or a lot of money. Because there's a lot of course creators that make a lot of money by putting those two words in the name of their course.”

“The attacker actually broke into Context AI, which is an AI tool that Vercel, one of their employees was using. And from there, they got into that employee's Google workspace and from there into Vercel's environment. So there's kind of like this multi-step approach that they took in order to get in.”

“The worst situation we can be in is when our job is just good enough to cover our bills. If they were to get a 50 percent cut, then that could be the best thing that ever happened to them. ... sometimes when the boats are burned for us, we talk about the metaphor of burning your boats, not looking back, no plan B. When that choice is taken from us, it can be the best thing that ever happens to us.”

“He said that AI is now generating more than 65% of new code at Snap and the company expects to cut over $500 million in annualized expenses by the back half of this year. So they are cutting in a massive way. The stock jumped almost 8% on the news, and I think that tells you exactly how Wall Street is feeling about this tradeoff.”

“I know the answer, the answer is freedom. Freedom is what brings the left and the right together, because it reduces the size of the state and the influence, reduces the malign influence of the media, it reduces the money printing, and within that, you have Bitcoin, which is a solution, which is great. But you cannot sell freedom to enough people, because when you sell freedom, you have to make compromises.”

“a real affinity now towards trade roles... especially kind of Gen Z sees as a much safer option. What do you mean trade uh like like firstline jobs um you know typical like trade roles not office jobs and they see those as more resilient in an AI world. These are the types of jobs that AI probably won't take. Uh so we're seeing more affinity towards that as well.”

“Imagine your favorite JavaScript library, like, let's say, React. Right, you you learn through all of your pretraining data that there is a function called x. But at some point, a new version of React comes out and turns out that it's a breaking change, and all of a sudden, x function doesn't exist. It's now a y function. No matter how much you say it in the context, you cannot just override what's the most intuitive throughout all of the model parameters to basically say x.”

“Any honest argument about continual learning pretty much has to start with in context learning because it genuinely works. We see that with examples like Karpathy's auto research project. Kind of like the other examples we give in the article is OpenCloud. Like, the underlying model was available to anyone, but, what's really made it a special magical, moment is, this kind of like orchestration of the context.”

“Don't share your ideas because people will copy them. That's total garbage. We should share ideas with everyone. ... I think what people don't understand is that when a new entrant joins an industry, in most cases, not all cases, it grows the entire industry, right? ... When you join a market, it grows the market. I knew a woman who made a million dollars decorating porches with pumpkins, right? That's all she does.”

“So the survey came out that we thought we were seventh. We are poorer than Mississippi, which is the poorest US state. There are things we outperform on. We do outperform like in certain measures that people are happy about, which is like health care.”

“It comes down to high gross margin. It's very, very hard to overcome a business that does not have high gross profit margin. And then leadership, like you can't, you put a good leader in a crappy business and you've got a good business. You can't sacrifice that. I don't spend a lot of time in interviews and like going back and forth and doing reference checks. I just look at this like a numbers game.”

“What's actually required for AGI is the ability to keep learning after training and the move from pattern matching to understanding cause and effect.”

“Liz Truss replied and said, well, this is a point with communists. You can't convince them. You have to defeat them. And so I was thinking about that in the lift, and I realized the only answer to this is we just have to go and kill them all.”

“But there needs to be a big enough need there, and they need to have a bias for action. Like they need to just be impatient. Impatience can be a superpower. They need to just do things, not just research endlessly, but be willing to go out there and try to get a customer. If they're not willing to do that, then they probably don't want to be an entrepreneur.”

“The test that some people use currently is pretty simple. You basically you train a model that is learned on x y z data. And once you deploy, you just want to check whether it learns something out of distribution, something that it hasn't seen before. And we are starting to see some examples like the test time training done by Yusan, with the discover paper that kind of makes some of the novel inventions.”

“We make this, like, very high level framework in terms of just the three buckets of the context, the modules, and also the weights. And the distinction that like, the one callout that I think is important is all of these are learning mechanisms. And even in context learning, it's still a form of continual learning. But context is essentially what we call nonparametric learning, where we don't actually update the weights.”

“Using natural language, just like we're using today, to build an app. Hey, build me a credit card dashboard where I can do travel hacking all in one place and see when my cards expire, so I don't pay the renewal fee on cards I'm not using, etc. One prompt to get one app. I mean, if you think about it, colleges are teaching students today what AI can already do better.”

“I think they should try to find a customer first, a paying customer on any idea. Just steal an idea, honestly, like a burger restaurant. People try to think of a new idea. New ideas are more likely to fail than anything because it's new. I want an idea with precedent, right? I want a roadmap of someone that's gone before me and done that, and then I just want to copy that.”

“We are no longer a high trust society. What we now have is a grift society. I'm going to walk into that shop, I'm going to take what I want, because I don't care because there's no consequence. I think the easiest way to understand when people say, what is British culture? I say, we've gone from a high trust, civil, respectful society to a low trust, grift, take what you want.”

“No. There's just a big asterisk by it. You have to work your way up to passive income, which takes a lot longer than most people think. ... So it takes active work to then create passive income at some point in the future. Yes, or money, or a lot of money. Because there's a lot of course creators that make a lot of money by putting those two words in the name of their course.”

“I've had it been doing AI on my website, SEO on my website. Okay. It went and screwed up five pages. So it went and did some optimization on the page, and decided to do it with the API. Anyway, I went on to the website and there were pages missing. So I said, did you delete them? They said, no. I was like, well, they're not there and you're the only one who's been working on it. It's like, no, it definitely wasn't me. It must be you. It's like, I think I'm a human. I think I would know it was me.”

“Previously, they had Image 1.5. Basically, it's really good at doing something that the last model, 1.5, was pretty bad at, which is rendering text. If you ask it for a poster, you know, it would always get this kind of like elfish looking, I don't know, the letters weren't that great. Anyways, 2 in their words is a step change.”

“And then everything changed when I went and served a two-year mission for my church. I went to Hungary. I learned Hungarian and I knocked doors and I approached people on the street. I was an extreme introvert for two years. And that just like completely rewired my brain to make me embrace failure, embrace embarrassment and just not care anymore.”

“I said to him, I said, this is what you should do. Don't start a company now, take that role. Learn on their time. And what I'll do with you is, if I were you, is I would go in there in the first three months, run a secret project, don't tell anyone else, learn how to replace you in that company with AI. Then go to your boss, say to your boss, I'd like to have a meeting. And in that meeting, put together a presentation and show them how you've learned to replace yourself with AI.”

“But there needs to be a big enough need there, and they need to have a bias for action. Like they need to just be impatient. Impatience can be a superpower. They need to just do things, not just research endlessly, but be willing to go out there and try to get a customer. If they're not willing to do that, then they probably don't want to be an entrepreneur.”

“Somebody walks into my house. This is a shoes-free house. Let's say somebody comes in tracking mud all over the place. That person's going to get dragged by their hair out and then they're never coming back in. Zero tolerance policy for broken windows. When these minor infractions are permitted, the Overton window, the broadness of what is now allowable behavior shifts. If you allow minor infractions, you're going to get moderate infractions. You allow those, you're going to get major infractions. From the very first days of the blog, the comments section has guidelines. If you're an asshole, we're going to boot you.”

“Let's just start with what is your career advice to young people? What do you talk about in the book?”