"a lot of people have said for many years that there will come a time when they want to pause a little bit that time is now the following is a conversation with Max tegmark his third time in the podcas..."

"specific and addresses a very small pool of actors who possesses this capability the letter has been signed by over 50 000 individuals including 1800 CEOs and over 1500 professors signatories include ..."

"all Max I just have to say uh thank you for giving me a chance thank you for starting this journey it's been an incredible journey just thank you for um sitting down with me and just acting like I'm s..."

"we look after the Stars if you define our universe the way most astrophysicists do not this all of space but the spherical region of space that we can see with our telescopes from which light has the ..."

"so uh we're going to give birth to an intelligent alien civilization unlike anything that human the evolution here on Earth was able to create in terms of the path the biological path it took yeah and..."

"I try but I would say I fail I mean it's it's very difficult for a human mind really grapple with something still completely alien even even for us right if we just try to imagine how would it feel if..."

"might actually feel that the the one thing which is so obvious though it's I think is just really worth reflecting on is because the mind space of possible intelligence is so different from ours it's ..."

"us meaning as well right so for example I found it so shocking that this new Microsoft gpt4 commercial that they put together has this woman talking about and showing this demo how she's going to give..."

"aspects of that but mostly it's just the transformation of how humans communicate and it's scary because so much of our society is based on this glue of communication and if that we're now using AI as..."

"wants it significantly obviously supersedes the performance of humans it will live in its own world and provide maybe a uh flourishing Civilizations for humans but we humans will continue hiking mount..."

"subjective experience is a fundamental value to what it means to be human make that make that the priority that feels like a hopeful direction to me but that also requires more compassion not just tow..."

"lifetime a great deal right so and um you know you were born without being able to speak English and at some point you decided hey I want to upgrade my software let's install an English-speaking modul..."

"pattern in the same way you Lex you're not the same atoms as during the first time you did with me you've swapped out most of them but still you yeah and the the information pattern is still there and..."

"even with Life 2.0 we can to some extent already transcend our physical bodies and our death and particularly if you can share your own information your own ideas with many others like you do in your ..."

"this um just feeling comfortable with not buying into what everybody else is saying just dude what I think is right they both very much just you know did their own thing and sometimes they got flagged..."

"and and even if everybody else keeps thinking it's there's a certain um I always root for the underdog when I watch movies and my my dad once I I one time for example when I wrote one of my craziest p..."

"thinking about you know this thing we're talking about which is humans creating another living sentient perhaps uh being I think it uh mainly do two things uh one of them just going through all their ..."

"know the other thing is dealing with death up in personal like this it's actually made me less afraid of um even less afraid of other people telling me that I'm an idiot you know which happens regular..."

"being grateful yeah well you do happen to be working on a thing which seems to have a potentially some of the greatest impact on human civilization of anything humans have ever created which is artifi..."

"that's about to hit us right uh most politicians don't even have their radar this on the radar they think maybe in 100 years or whatever right now we're at a fork on the road this is the most importan..."

"ourselves as Homo sapiens because it seemed like the basic thing where the King of the castle on this planet were the Smart Ones if we can control everything else this could very easily change we're c..."

"AGI AI General AI systems and what kind of impact that has on society exactly when that thing is achieves General human level intelligence and then beyond that General superhuman level intelligence th..."

"hard on 2014-2015 was the mainstream AI safety the idea that there even could be risks and that you could do things about them before then a lot of people thought it was just really kooky to even talk..."

"um mechanistic interpretability in geek speak you know you have this machine it does something smart you try to reverse reverse engineer see how does it do it are you think of it also as artificial Ne..."

"incredibly dumb for for any fellow nerds listening to this there was a big Matrix and a and roughly speaking there are certain row and column vectors which encode these things and the they correspond ..."

"and it seems like the scary thing about the effectiveness of large language models uh so Sam Altman every conversation with and he really showed that the leap from gpt3 to gpt4 has to do with just a b..."

"again what this open letter is calling for is just pausing all training of uh systems that are more powerful than gpt4 for six months let's give a chance for the labs to coordinate a bit on safety and..."

"right most a good fraction of all the bad things that we humans do are caused by moloch and I I like uh Scott Alexander's um naming of the monster so we can we humans can think of it as an f a thing i..."

"everywhere and um molok is not a new arrival on on the scene either we humans have developed a lot of collaboration mechanisms to help us fight back against Malik through various kinds of constructive..."

"matter how good-hearted they are because Malik is a really powerful foe so the idea is to for the major developers of AI systems like this so we're talking about Microsoft Google meta and anyone else ..."

"just need you need a public awareness of the of what the risks are and the broader Community coming in and saying hey let's slow down and you know another another common pushback I get today is we We ..."

"it's I'll say this again because this is this very basic point I think a lot of people get wrong because a lot of people dismiss the whole idea that AI can really get very superhuman because they thin..."

"suicide race you know it's um it's kind of like we're rushing towards this cliff but the closer to the cliff we get the more Scenic the views are and the more money there is there and the more so we k..."

"this thing and the hope you have is that there's some lower level of development which will not which will not allow us to lose control this is an interesting thought you have about losing control so ..."

"this is out of control Express train we're on now is gonna get the AGI that's why we need a little more time I feel is there something to be said well like Sam Allman talked about which is while we're..."

"and uh another thing high risk is connected to the internet Let It Go to websites download stuff on its own and talk to people oops we've done that already you know Elias yukowski you said you intervi..."

"media I'll recommender algorithms which do exactly that they they get so good at knowing us and pressing our buttons that we've we're starting to create a world now where we just have ever more hatred..."

"and that is not something you think is a new man in his interest yes right now it's mainly letting some humans manipulate other humans for profit and Power which is already caused a lot of damage and ..."

"a less creepy algorithm because then they would lose out on our Revenue to the other company is there any way to win that battle back just if we just Linger on this one battle that we've lost in terms..."

"between someone doing good things for for Humanity and bad things is not some sort of fairy tale thing that this person was born with the evil Gene and this one was not born with a good Gene no I thin..."

"we were Neanderthals would we really consider it a smart move if the if we had really Advanced biotech to build homo sapiens you you know you might say hey Max you know yeah let's build build the thes..."

"it's going to be fine and and besides look at these babies they're so cute clearly they're totally harmless that's exact those babies are exactly gpt4 yeah it's not I want to be clear it's not gpt4 th..."

"these work under the hood they're like the minimum viable intelligence they do everything you know the dumbest way that still works sort of yeah and um so they were life 3.0 except when they replace t..."

"but I you still need a human in the loop to kind of manage the design of things manage like what are the prompts that generate the kind of stuff to do some basic adjustment of the codes let's do some ..."

"I yeah I feel we have to remember as individuals we should from time to time ask ourselves why are we doing what we're doing all right then as a species we need to do that too so if we're building as ..."

"choice but to fall for it right come also the thing you said about you using uh co-pilot AI tools to program faster how many time what factor faster would you say your code now does it go twice as fas..."

"you said there is just an exact example of these sort of things another thing which which um I was kind of lying on my psychiatrist imagining I'm on a psychiatrist's couch here saying what are my fear..."

"don't want to pick on any particular companies I think they're all under a lot of pressure to make money yeah and um again we the reason we're calling for this pause is to give them all cover to do wh..."

"nuclear reaction that if H1 can make more than one then you get an exponential growth in that we call it a nuclear explosion all explosions are like that and an intelligence explosion it's just exactl..."

"bigger different bigger jump in intelligence than if you go from uh from an ant to a human I think of course what we want to do is have have a controlled thing the nuclear reactor you put moderators i..."

"we have and and I don't believe in this just asking people to look into their hearts and do the right thing it's easier for others to say that but like if if you're in a situation where your company i..."

"aligned with what's good for the greater group or part of right and then uh as we got a bit more sophisticated and developed language we invented gossip which is also a fantastic anti-malark right bec..."

"Regulators been able to keep up right so Regulators it's kind of comical the European Union right now is doing this AI Act Right which and in the beginning they had a little opt-out exception that gpt..."

"this just shows you know monologue can be defeated but the the challenge we're facing is that the tech is generally much faster than what the policymakers are and a lot of the policy makers also don't..."

"and but I think that the vision the path to success I see is first you give a breather actually to to the people in these companies their leadership who wants to do the right thing and they all have s..."

"to get policy makers to to see yeah this is a good idea and it's it's you know for the fight for the companies to fight molok they want and I I believe Sam Altman has explicitly called for this they w..."

"technology has gotten so good so fast become powerful to a degree where you could see in the near term the ability to make a lot of money and to put guard rails develop guard rails quickly in that kin..."

"who build this stuff they all realize that intelligence is information processing of a certain kind and it really doesn't matter at all whether the information is processed by carbon atoms in neurons ..."

"forget about geopolitical competition and just focus on the real issue that's why we put this that's really interesting but you've already made the case that uh even for China if you actually want to ..."

"lose you know if you if we can see in history once you have some species or some group of people who aren't needed anymore it doesn't usually work out so well for them right yeah there were a lot of h..."

"everybody's equal and the government will always will always protect them but if you look in practice groups that are very disenfranchised and don't have any actual power it usually gets screwed and u..."

"yesterday on social media I saw about this guy who was doing 3D modeling for gaming and if and all of a sudden now they got this new software you just give says prompts and he feels his whole job that..."

"you know I I think it's super cool um so we we need to stop and ask ourselves why again are we doing this as humans like I feel that AI should be built by Humanity for Humanity and let's not forget th..."

"the Act of Creation you you have an idea you design it and then you bring it to life and it does something especially if there's some intelligence that it does something it doesn't even have to have i..."

"it is for uh the the 3D modeler no longer being able to really enjoy the art of modeling uh 3D things for gaming I don't know I don't know what to make sense of that maybe I would ReDiscover that the ..."

"off than today including the richest people can be better off as well right it's not a zero-sum game you know technology again you can have two countries like Sweden and Denmark had all these ridiculo..."

"default outcome that why income inequality keeps going up that's why the life expectancy in the US has been going down now I think it's four years in a row I just read a heartbreaking study from CDC a..."

"what if you flip this around and this is an idea I should give credit to Steve I'm 100 for so that it will only run the code if it can prove instead of not running it if it can prove that it's not tru..."

"and that leap is when it runs away yeah I I don't buy that argument I think no matter how super intelligent an AI is it's never going to be able to prove to me that they're only finitely many primes f..."

"for you yeah there are other things too that we shouldn't sweep under the rug like not every human agrees on exactly where what direction we should go with Humanity right yes and you've talked a lot a..."

"that warlock wins because frankly molok going wild now it feels like a war on Life playing out in front of our eyes if you if you just look at it from space you know we're on this planet beautiful vib..."

"and then we're giving ever more power to things which are not alive these large corporations are not living things right they're just maximizing profit there I want to win them more in life I I think ..."

"had a little baby and I keep asking myself you know is um um how old is he even gonna get you know and and um I ask myself hey it feels I said to my wife recently it feels a little bit like I was just..."

"really if the stakes are really high they can prove to us that they're safe it's really just virus checking in Reverse again I I think it's scientifically doable I don't think it's hopeless um we migh..."

"can be right we talked a lot about stuff on Earth it can be great but even if you really get ambitious and look up at the skies right there's no reason we have to be stuck on this planet for the rest ..."

"source a technology technology yeah is there something to be said about how to get the release of such systems right like GPT 4 and gpt5 so open AI went through a pretty rigorous effort for several mo..."

"uh is there something you could say about ideas you have it's just on looking having thought about this problem of AI safety how to release such system how to test such systems when you have them insi..."

"the room and and uh obviously no matter how hard any of these companies have tried uh they that's not something that's easy for them to verify with large language models and the only way to be really ..."

"make a lot a ton of money on them you know uh we're gonna just watch now this coming year right Microsoft is rolling out the new uh office suite where you go into Microsoft Word and give it a prompt t..."

"where it becomes much easier for very few to kill very many and therefore it becomes very easy for very few to dominate very many right um yeah if you want to know how I could kill all people just ask..."

"right so if um if you're in Ai and you just wanna figure something out then you decide you know we just really need them the space here to build more compute facilities you know if that's the only goa..."

"a human child first they're just not smart enough to understand our goals we can't even talk and then eventually they're teenagers and understand our goals just fine but they don't share yeah but ther..."

"one Lex that aligned AI can help us solve all the other problems because it seems like it has to have constant humility about his goal constantly question the goal because as you optimize towards a pa..."

"there are a lot of brilliant people now working on AI safety and we're just not we just need to give them a bit more time but also not that many relative to the skill of the problem no way exactly the..."

"changing now thanks to the gpt4 right so I I think if there's a silver lining to um what's happening here even though I think many people would wish it would have been rolled out more carefully is tha..."

"information you can go around and around in in computer science nerdspeak you call it a recurrent neural network where some of the output gets fed back in again and with his mathematical formalism if ..."

"if if tenoni is right and it's all these are all trans Transformers that are not in the sense of the of Hollywood but in the sense of these one-way Direction and neural networks so they're all the zom..."

"humans will create something that is better than us humans in the in the way that we find beautiful which is they they cut they have a deeper subjective experience of reality not only are they smarter..."

"take on this challenge is there something you could say to the timeline that you think about uh about the development of AGI depending on the day I'm sure that changes for you but when do you think th..."

"happening very soon that's why we decided to do this open letter because you know if there's ever been a time to pause uh you know it's today there's a feeling like this gpt4 is a big transition into ..."

"yeah what do you learn about human nature from that it's our old friend moloch again it's really scary to see it where America doesn't want there to be a nuclear war Russia doesn't want to be a global..."

"either like you remember when Argentina invaded the Falkland Islands the the military Junta that ordered that right people were cheering on the streets at first when they are when they took it and the..."

"worried is not just that I it's difficult to see an ending a quick peaceful ending to this tragedy that doesn't involve some horrible escalation um but also that we understand more clearly now just ho..."

"so you know I'm not good at farming but uh it's knowing if it drops below freezing pretty much almost days in July and then like that's not good so they worked out they put this into their farming mod..."

"and um and starving to death is is uh the worst way to die as Audemars all all the famines in history show the torture involved in that probably brings out the worst in people also when when people ar..."

"yeah yeah we are fighting Malik for human survival we as a civilization have you seen the movie Needful Things it's a Stephen King novel I love Stephen King and uh Max fonseedov Swedish actors playing..."

"what kind of what would you ask her what kind of question would you ask what would you talk about something so much smarter than you a really Zinger of a question that's a good one would you be afraid..."

"terms of physics how little how little we humans understand I'm cool with that I think I think um so I I can't waffle my way out of this question it's a fair one it was tough I think given that I'm a ..."

"and I do you want me to give you a hand wavy argument for it you know like in my lab again every time we look at how it how these large language models do something we see that they do them in really ..."

"if you look at our brains actually our brains are part zombie and part conscious when I open my eyes I immediately take all these pixels and hit my ref on my retina right and like oh that's Lex but I ..."

"same thing that makes us uh special yeah Max uh it's a huge honor that you will sit down to me the first time uh on the first episode of this podcast it's a huge honest sit down with me again and talk..."