Library version changes expose limits of context-only learning
“Imagine your favorite JavaScript library, like, let's say, React. Right, you you learn through all of your pretraining data that there is a function called x. But at some point, a new version of React comes out and turns out that it's a breaking change, and all of a sudden, x function doesn't exist. It's now a y function. No matter how much you say it in the context, you cannot just override what's the most intuitive throughout all of the model parameters to basically say x.”
Out-of-distribution learning post-deployment is the key milestone
“The test that some people use currently is pretty simple. You basically you train a model that is learned on x y z data. And once you deploy, you just want to check whether it learns something out of distribution, something that it hasn't seen before. And we are starting to see some examples like the test time training done by Yusan, with the discover paper that kind of makes some of the novel inventions.”
“Any honest argument about continual learning pretty much has to start with in context learning because it genuinely works. We see that with examples like Karpathy's auto research project. Kind of like the other examples we give in the article is OpenCloud. Like, the underlying model was available to anyone, but, what's really made it a special magical, moment is, this kind of like orchestration of the context.”
AI models today are frozen like the amnesiac in Memento
“The main protagonist, Leonor Shelby, has a form of this amnesia where he cannot form new memories. So he goes about his life, with kind of, like, this cut off date after which point he has kind of these long term memories, but really cannot retain anything new that he experiences. And so what he does is he uses the sticky notes where he writes some of the notes to himself. He pulls out his Polaroid camera to capture the moments as he goes on about his life. And, I mean, he even tattoos some of the, memories that he wants to imprint in his memory.”
Adversarial security requires updating weights, not prompts
“The first one is essentially in adversarial security. Like, imagine there is a new jailbreak attack. You have your model deployed in the wild, and it's being used. Imagine you try to update your system prompt to say, like, don't do this. Like, it's not going to work. Right? Because all of the, like, parameters in the model have learned to be helpful to the users. So you really have to encompass that kind of knowledge in the weights where the attackers don't have access to.”
Learning happens across context, modules, and weights
“We make this, like, very high level framework in terms of just the three buckets of the context, the modules, and also the weights. And the distinction that like, the one callout that I think is important is all of these are learning mechanisms. And even in context learning, it's still a form of continual learning. But context is essentially what we call nonparametric learning, where we don't actually update the weights.”
The ultimate test is models that learn on the job like humans
“We humans are not AGI, but we still learn on the job. We learn from experience, and that's what makes kind of humans kind of unique. And so that's kind of like the ultimate test. Like, how do we define that that we got to continue learning? It's like, well, is there a system that is able to learn on the job and get better through use just like humans?”
