"if you extrapolate the curves that we've had so far right if if you say well I don't know we're starting to get to like PhD level and and last year we were at undergraduate level and the year before w..."

"here's Dario amade let's start with a big idea of scaling laws and the scaling hypothesis what is it what is its history and where do we stand today so I can only describe it as it you know as it rela..."

"at at every stage of scaling there are always arguments and you know when I first heard them honestly I thought probably I'm the one who's wrong and you know all these all these experts in the field a..."

"in terms of expertise in physics uh there's this there's this concept called the one over F noise and one overx distributions um where where often um uh you know just just like if you add up a bunch o..."

"natural question then is what's the ceiling of this like how complicated and complex is the real world how much of stuff is there to learn I don't think any of us knows the answer to that question um ..."

"but there is a balance if we do hit a limit if we do hit a Slowdown in the scaling laws what do you think would be the reason is it compute limited data limited uh is it something else idea limited so..."

"perhaps could be one reason what about the limits of compute meaning uh the expensive uh nature of building bigger and bigger data centers so right now I think uh you know most of the Frontier Model c..."

"you know anthropics anthropic mission is to kind of try to make this all go well right and and you know we have a theory of change called race to the top right race to the top is about trying to push ..."

"interesting thing is that as we've done this other companies have started doing it as well in some cases because they've been inspired by it in some cases because they're worried that uh you know if i..."

"right thing and it's not it's not about us in particular right it's not about having one particular good guy other companies can do this as well if they if they if they join the race to do this that's..."

"can talk in much more detail about this to Chris that when we open them up when we do look inside them we we find things that are surprisingly interesting and as a side effect you also get to see the ..."

"because it was taken down I think after a day somehow these interventions on the model um where where where where you kind of adjust Its Behavior somehow emotionally made it seem more human than any o..."

"about the present let's talk about Claude so this year A lot has happened in March claw 3 Opa Sonet Hau were released then claw 35 Sonet in July with an updated version just now released and then also..."

"to use the model very broadly so we wanted to serve that whole spectrum of needs um so we ended up with this uh you know this kind of poetry theme and so what's a really short poem it's a Haik cou and..."

"is about as good as Opus 3 the largest old model so basically the aim here is to shift the curve and then at some point there's going to be an opus 3.5 um now every new generation of models has its ow..."

"processes um uh there's pre-training which is you know just kind of the normal language model training and that takes a very long time um that uses you know these days you know tens you know tens of t..."

"course you know we're always trying to make the processes as streamlined as possible right we want our safety testing to be rigorous but we want it to be RoR ous and to be you know to be automatic to ..."

"incredible discoveries like they they they they they almost always come down to the details um and and often super super boring details I can't speak to whether we have better tooling than than other ..."

"RF it's a bunch of other methods as well um post training I think you know it's becoming more and more sophisticated well what explains the big leap in performance for the new Sona 35 I mean at least ..."

"increase in kind of in kind of programming programming ability and and I would suspect that if we can get to you know 90 90 95% that that that that you know it will it will represent ability to autono..."

"kind of break out of the break out of the scheme it's not like software where you can say oh this is like you know 3.7 this is 3.8 no you have models with different different tradeoffs you can change ..."

"of training the models so from the user side the user experience of the updated Sonet 35 is just different than the previous uh June 2024 Sonet 35 it would be nice to come up with some kind of labelin..."

"we want models to have have and which we don't want to have that itself the normative question is also super interesting I got to ask you a question from Reddit from Reddit oh boy you know there there..."

"it wouldn't certainly in the current setup it would not make sense to do that now there are a couple things that we do occasionally do um one is sometimes we run AB tests um but those are typically ve..."

"um I I think it actually relates to one of the things I said before which is that models have many are very complex and have many aspects to them and so often you know if I if I if if I ask a model a ..."

"work this is such a piece of crap exactly so it's easy to have the conspiracy theory of they're making Wi-Fi slower and slower this is probably something I'll talk to Amanda much more about but U anot..."

"was a period during which models ours and I think others as well were T verbose right they would like repeat themselves they would say too much um you can cut down on the verbosity by penalizing the m..."

"that we can start to study today right I think I think that that that difficulty in in steering the behavior and in making sure that if we push an AI system in One Direction it doesn't push it in anot..."

"future what's the current best way of gathering sort of user feedback like uh not anecdotal data but just large scale data about pain points or the opposite of pain points positive things so on is it ..."

"being a a much more intelligent agent AI could break that correlation and so I I I I I do have serious worries about that I believe we can prevent those worries uh but you know I I think as a Counterp..."

"very fast uh so the solution we came up with for that in in collaboration with uh you know people like uh the organization meter and Paul Christiano is okay what what what what you need for that or yo..."

"if then structure which is if the models pass a certain capability then we impose a certain set of Safety and Security requirements on them so today's models are what's called asl2 models that were a ..."

"enough so asl3 is going to be the point at which uh the models are helpful enough to enhance the capabilities of non-state actors right State actors can already do a lot a lot of unfortunately to a hi..."

"it's it's some some some amount of acceleration in AI research capabilities with an with an AI model and then asl5 is where we would get to the models that are you know that are that are kind of that ..."

"dangers are here what do you think the timeline for asl3 is where several of the triggers are fired and what do you think the timeline is for asl4 yeah so that is hotly debated within the company um u..."

"are are I won't say straightforward they're they're they're they're rigorous but they're easier to reason about I think once we get to asl4 um we start to have worries about the models being smart eno..."

"exotic ways you can think of that it might also not be reliable like if the you know model gets smart enough that it can like you know jump computers and like read the code where you're like looking a..."

"works uh and where that's headed yeah it's actually relatively simple so Claude has has had for a long time since since Claude 3 back in March the ability to analyze images and respond to them with te..."

"releasing releasing the model while while while the capabilities are are you know are are still are still limited is is is very helpful in terms of in terms of doing that um you know I think since it'..."

"you know super capable yeah and there's uh a lot of interesting attacks like prompt injection because now you've widened the aperture so you can prompt inject through stuff on screen so if this become..."

"today yeah the science of building a box from which asl4 AI system cannot Escape I I think it's probably not the right approach I think the right approach instead of having something you know unaligne..."

"think you can trust these companies to adhere to these voluntary plans in their own right I like to think that anthropic will we do everything we can that we will our our our our RSP is checked by our..."

"terms of addressing the risks um you don't really hear about it on Twitter you just hear about kind of you know people are people are cheering for any regulation and then the folks who are against mak..."

"capture they're not sci-fi fantasies they're not they're not any of these things um you know every every time we have new model every few months we measure the behavior of these models and they're get..."

"about the different players in the game one of the uh ogs is open AI you have had several years of experience at open AI what's your story and history there yeah so I was at open AI for uh for roughly..."

"I spent there I think I had a particular vision of how these how we should handle these things how we should be brought out in the world the kind of principles that the organization should have and lo..."

"again at the end it's it's not about one company winning or another company winning if if we or another company are engaging in some practice that you know people people find genuinely appealing and I..."

"could happen then then it doesn't matter which company was ahead um if instead you create a race to the top where people are competing to engage in good in good practices uh then you know at at the en..."

"regulation you said Talent density beats Talent Mass so can you explain that can you expand on it can you just talk about what it takes to build a great team of AI researchers and Engineers this is on..."

"we how we grow uh early on and and now as well you know we've hired a lot of physicists um you know theoretical physicists can learn things really fast um uh even even more recently as we've continued..."

"know some tiny number of people some singled digigit number of people have have driven forward the whole field by realizing this uh and and it's you know it's often like that if you look back at the D..."

"let's talk if we could a bit about posttraining yeah so it uh seems that the modern posttraining recipe has uh a little bit of everything so supervised fine tuning rhf uh the the the Constitutional AI..."

"invent okay well about let me ask you about specific techniques so first on rhf what do you think think just zooming out intuition almost philosophy why do you think rhf works so well if I go back to ..."

"can say in terms of cost is pre-training the most expensive thing or is post-training creep up to that at the present moment it is still the case that uh pre-training is the majority of the cost I don..."

"principles that the models uh you know have to obey I think a lot of them are things that people would agree with everyone agrees that you know we don't you know we don't want models to present these ..."

"that's a pretty useful direction again it has a lot in common with uh constitutional AI so again another example of like a race to the top right we have something that's like we think you know a bette..."

"a long one it is rather long yeah it's really refreshing to read concrete ideas about what a positive future looks like and you took sort of a bold stance because like it's very possible you might be ..."

"rational that line of reasoning that I just gave might be um if if you kind of only talk about risks your brain only thinks about risks and and so I think it's actually very important to understand wh..."

"very serious about anything that could derail them so I think the starting point is to talk about what this powerful AI which is the term you like to use uh most of the world uses AGI but you don't li..."

"intelligence so it's uh both in creativity and be able to generate new ideas all that kind of stuff in every discipline Nobel Prize winner okay in their prime it can use every modality it so uh that's..."

"solve this abstract differential equation then like 5 days after we you know we build the first AI That's more powerful than humans then then uh you know like the world will be filled with these AIS a..."

"just been it's been very difficult it's also been very difficult to get you know very simple things through the regulatory system right I think you know and you know I I don't want to just spage anyon..."

"this the case people point to the structure of firms the structure of Enterprises how um uh you know how slow it's been to roll out our existing technology to very poor parts of the world which I talk..."

"moderately fast is that you talk to what I find is I find over and over again again in large companies even in governments um which have been actually surprisingly forward leaning uh you find two thin..."

"it's a balanced fight between the two because inertia is very powerful but but but eventually over enough time the Innovative approach breaks through um and I've seen that happen I've seen the Arc of ..."

"think a lot of these people who write down the differential equations who say AI is going to make more powerful AI who can't understand how it could possibly be the case that these things won't won't ..."

"can just and it's something we should all be able to agree on right like as much as we fight about about all these political questions is is this something that could actually bring us together um but..."

"clusters as much as we want like you know maybe Taiwan gets blown up or something and you know then we can't produce as many gpus as we want so there there are all kinds of things that could could der..."

"so you extensively describe sort of the compressed 21st century how AGI will help uh set forth a chain of breakthroughs in biology and medicine that help us in all these kinds of ways that I mentioned..."

"of that and so when I think of like where will AI have an impact I'm like can AI turn that small fraction into a much larger fraction and raise its quality and within biology my experience within biol..."

"selectively change things um and and my view is that there's so much more we can still do there right you can do crisper but you can do it for your whole body um let's say I want to do it for one part..."

"the next experiment is I'm going to like write some code and run a statistical analysis all the things a grad student would do there will be a computer with an AI that like the professor talks to ever..."

"gone from 3% in January of this year to 50% in October of this year so you know we're on that S curve right where it's going to start slowing down soon because you can only get to 100% but uh I you kn..."

"less writing things line by line and it'll be more macroscopic and I wonder what the future of Ides looks like so the tooling of interacting with AI systems this is true for programming and also proba..."

"meaning I worry about economics and the concentration of power that's actually what I worry about more um I I worry about how do we make sure that that fair World reaches everyone um when things have ..."

"for listening to this conversation with Dario amade and now dear friends here's Amanda Asal you are a philosopher by training so what sort of questions did you find fascinating through your journey in..."

"think it would be pretty inspiring for people that are quote unquote non-technical to see where like The Incredible Journey you've been on so what advice would you give to people that are sort of mayb..."

"claude's position so imagine that I take someone and they're they know that they're going to be talking with potentially millions of people so that what they're saying can have a huge impact um and yo..."

"you don't need an MRI that's a good person to listen to um and like it's actually really nuanced what you should do in that kind of case because you also want to be like but if you're trying to advoca..."

"lots of different political views lots of different ages um and so you have to ask yourself like what is it to be a good person in those circumstances is there a kind of person who can like travel the..."

"so like I said you had a lot of conversations with Claude can you just map out what those conversations are like what are some memorable conversations what's the purpose the the goal of those conversa..."

"like if I was a company making prompts for models I'm just like in if you're willing to spend a lot of like time and resources on the engineering behind like what you're building then the prompt is no..."

"what kind of like what coding language you wanted to use is that because like it was just very ambiguous and it it kind of had to take a guess in which case next time you could just be like hey make s..."

"into the technical for a little bit so uh the magic of post training y why do you think rhf works so well to make the model seem smarter to make it more interesting and useful to talk to and so on I t..."

"the other side of PSE training this really cool idea of constitutional AI you're one of the people that critical to creating that idea yeah I worked on it can you explain this idea from your perspecti..."

"likely to uh like encourage people to purchase illegal weapons like that's probably a fairly specific principle but you can give any number um and the model will give you a kind of ranking and you can..."

"so you might have a principle that's like imagine that the model um was always like extremely dismissive of I don't know like some political or religious view for whatever reason like so you're like o..."

"almost like the less robust but faster way of just like solving problems let me ask about the feeling of intelligence so Dario said that Claude any one model of Claude is not getting Dumber MH but the..."

"one of I guess the things to remember here is the that just the details of a prompt can have a lot of impact right there's a lot of variability in the result and you can get Randomness is like the oth..."

"get like signal feedback about The Human Experience across thousands tens of th hundreds of thousands of people like what their pain points are what feels good are you just using your own intuition as..."

"important um and then yeah with the apologetic Behavior I don't like that and I like it when Claude is a little bit more willing to like push back against people or just not apologize part of me is li..."

"it's like one of those things where I'm like I do want it to get better but also while remaining aware of the fact that there's errors on the other side that that are possibly worse I think that matte..."

"that I usually ask a dumb question and you're like oh yeah that's a good question it's that whole vibe or I'll just misinterpret it and be like oh go with it I love it yeah I mean I have two thoughts ..."

"perfectly aligned with every human being and aggregate somehow um it's much more like let's make things like work well enough that we can improve them yeah generally I don't know my gut says like empi..."

"the floor um and so maybe that's like uh this this degree of like empirism and practicality comes from that perhaps to take a tangent on that since remind me of a blog post you wrote on optimal rate o..."

"think similarly where I'm like if the failures are small and the costs are kind of like low then I'm like then you know you're just going to see that like when you do the system prompt you can't it it..."

"know you're just like okay great like then when I go and I think about this I'll be like I'm maybe I'm not under failing in this area cuz like that one just didn't work out and from The Observer persp..."

"it's like uh like regardless of like whether there's anything behind it um it doesn't feel great do you think uh llms are capable of Consciousness H great and hard question uh coming from philosophy I..."

"of like calculation like it's really easy for people to think of the zero some cases and I'm like let's exhaust the areas where it's just basically Costless um to uh assume that if this thing is suffe..."

"like if someone is like hey I get a lot out of chatting with this model um I'm aware of the risks I'm aware it could change um I don't think it's unhealthy it's just you know something that I can chat..."

"relating to it doesn't solve everything but I think it helps quite anthropic may be the very company to develop a system that we definitively recognize as AGI and you very well might be the person tha..."

"not and obviously we see these with this with like more kind of like you see novel Solutions all the time especially to like easier problems I think people overestimate you know novelty isn't like is ..."

"especially the you know there's all this work on probing which you might see as part of being mechanistic interality although it's you know again it's just a broad term and and not everyone who does t..."

"so this is actually is indeed a a really remarkable and exciting thing where it does seem like at least to some extent you know the same the same elements the same the same features and circuits form ..."

"thing if that's true um you know it suggests that um well I think the thing that it suggests is the gradi scent is sort of finding you know the right ways to cut things apart in some sense that many s..."

"of Concepts that are formed yeah and like maybe there are ways to go and describe you know images without reference to those things right but they're not the simplest way or the most economical way or..."

"understand these models is in terms of neurons you could try to be like oh you know there's a dog detecting neuron and um here's a car detecting neuron and it turns out you can actually ask how those ..."

"worth trying to pin down like what what really um is the the core hypothesis here I think the the core hypothesis is something we call the linear representation hypothesis so um if we think about the ..."

"breakthroughs I think yeah well and actually this is another thing that I think is really interesting so um you know there a way in which I think it can be really useful for society to have people um ..."

"another interesting hypothesis is the superposition hypothesis can you describe what superos is yeah so earlier we were talking about word toac right and we were talking about how you know maybe you h..."

"it's really not the core thing that it's expecting right so if you look at a a curve detector for instance and you look at the places where it's 5% active you know you could interpret it just as noise..."

"some of when we're talking about neurons you can have many more Concepts than you have have neurons so that's the at a high level super hypothesis now it has this even Wilder implication which is um t..."

"to go and run conven on your GPU which does you know nice dense Matrix multiplies um and that you just can't beat that how many Concepts do you think can be shoved in into a neural network depends on ..."

"phenomenon and super is is a hypothesis that um would explain it along with with some other so that makes Mech turb more difficult right so if you if you're trying to understand things in terms of ind..."

"recent work has been aiming at is how do we extract the mod semantic features from a neural net that has politic features and all this this mess yes we have the have we observe these polyur and we hyp..."

"and um you know I should mention that there was this cunning home at all um that had very similar results around the same time there's something fun about being doing these kinds of small scale experi..."

"so you know this this is the only thing this is doing is that sort of um unfolding things for you so if everything was sort of folded over top of it you know cation folded everything on top of itself ..."

"that's a general challenge it's like it's it's St an incredible colish they can say a true thing but it doesn't it's qu it's not it's missing the depth sometimes and in this context it's like the arc ..."

"yeah I mean especially that's hilarious especially as we talk about AI safety and it looking for features that would be relevant to AI safety like deception and so on uh so let's let's talk about the ..."

"can yeah so maybe maybe let's start with a one example to start which is we found some features around sort of security vulnerabilities and back doors and codes so it turns out those are actually two ..."

"learned from detecting lying inside models yeah so I think we're in some ways in early days for that we find quite a few features related to deception and lying there's one feature where fires for peo..."

"a kind of dark matter and in not in maybe the sense of of astronomy of earlier astronomy when we didn't know what this unexplained matter is um and so I I think a lot about that that dark matter and w..."

"what do you think is the difference between the human brain the biological neuron Network and the artificial neuron Network well the neuroscientists have a much harder job than us you know sometimes I..."