In terms of the depth and range of topics, this episode is the best I’ve done. No part of my worldview is the same after talking with Carl Shulman. He's the most interesting intellectual you've never heard of. We ended up talking for 8 hours, so I'm splitting this episode into 2 parts. This part is about Carl’s model of an intelligence explosion, which integrates everything from: * how fast algorithmic progress & hardware improvements in AI are happening, * what primate evolution suggests about the scaling hypothesis, * how soon before AIs could do large parts of AI research themselves, and whether there would be faster and faster doublings of AI researchers, * how quickly robots produced from existing factories could take over the economy. We also discuss the odds of a takeover based on whether the AI is aligned before the intelligence explosion happens, and Carl explains why he’s more optimistic than Eliezer. The next part, which I’ll release next week, is about all the specific mechanisms of an AI takeover, plus a whole bunch of other galaxy brain stuff. Maybe 3 people in the world have thought as rigorously as Carl about so many interesting topics. This was a huge pleasure. Watch Part 2 here: https://youtu.be/KUieFuV1fuo 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒 * Transcript: https://www.dwarkeshpatel.com/carl-shulman * Apple Podcasts: https://bit.ly/3P9rPpJ * Spotify: https://bit.ly/42Vnbzb * Follow me on Twitter: https://twitter.com/dwarkesh_sp * Carl's blog: http://reflectivedisequilibrium.blogspot.com/ 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 - Intro 00:00:47 - Intelligence Explosion 00:17:18 - Can AIs do AI research? 00:38:15 - Primate evolution 01:02:45 - Forecasting AI progress 01:33:35 - After human-level AGI 02:08:54 - AI takeover scenarios
In terms of the depth and range of topics, this episode is the best I’ve done. No part of my worldview is the same after talking with Carl Shulman. He's the most interesting intellectual you've never heard of. We ended up talking for 8 hours, so I'm splitting this episode into 2 parts. This part is about Carl’s model of an intelligence explosion, which integrates everything from: * how fast algorithmic progress & hardware improvements in AI are happening, * what primate evolution suggests about the scaling hypothesis, * how soon before AIs could do large parts of AI research themselves, and whether there would be faster and faster doublings of AI researchers, * how quickly robots produced from existing factories could take over the economy. We also discuss the odds of a takeover based on whether the AI is aligned before the intelligence explosion happens, and Carl explains why he’s more optimistic than Eliezer. The next part, which I’ll release next week, is about all the specific mechanisms of an AI takeover, plus a whole bunch of other galaxy brain stuff. Maybe 3 people in the world have thought as rigorously as Carl about so many interesting topics. This was a huge pleasure. Watch Part 2 here: https://youtu.be/KUieFuV1fuo 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒 * Transcript: https://www.dwarkeshpatel.com/carl-shulman * Apple Podcasts: https://bit.ly/3P9rPpJ * Spotify: https://bit.ly/42Vnbzb * Follow me on Twitter: https://twitter.com/dwarkesh_sp * Carl's blog: http://reflectivedisequilibrium.blogspot.com/ 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 - Intro 00:00:47 - Intelligence Explosion 00:17:18 - Can AIs do AI research? 00:38:15 - Primate evolution 01:02:45 - Forecasting AI progress 01:33:35 - After human-level AGI 02:08:54 - AI takeover scenarios
The second half of my 7 hour conversation with Carl Shulman is out! My favorite part! And the one that had the biggest impact on my worldview. Here, Carl lays out how an AI takeover might happen: * AI can threaten mutually assured destruction from bioweapons, * use cyber attacks to take over physical infrastructure, * build mechanical armies, * spread seed AIs we can never exterminate, * offer tech and other advantages to collaborating countries, etc Plus we talk about a whole bunch of weird and interesting topics which Carl has thought about: * what is the far future best case scenario for humanity * what it would look like to have AI make thousands of years of intellectual progress in a month * how do we detect deception in superhuman models * does space warfare favor defense or offense * is a Malthusian state inevitable in the long run * why markets haven't priced in explosive economic growth * & much more Carl also explains how he developed such a rigorous, thoughtful, and interdisciplinary model of the biggest problems in the world. 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒 * Catch part 1 here: https://youtu.be/_kRg-ZP1vQc * Transcript: https://www.dwarkeshpatel.com/carl-shulman-2 * Apple Podcasts: https://bit.ly/3r1HBJk * Spotify: https://bit.ly/437t50c * Follow me on Twitter: https://twitter.com/dwarkesh_sp * Carl's blog: http://reflectivedisequilibrium.blogspot.com/ 𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒 This episode is sponsored by 80,000 hours. To get their free career guide (and to help out this podcast), please visit 80000hours.org/lunar. 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 - Intro 00:00:47 - AI takeover via cyber or bio 00:32:27 - Can we coordinate against AI? 00:53:49 - Human vs AI colonizers 01:04:55 - Probability of AI takeover 01:21:56 - Can we detect deception? 01:47:25 - Using AI to solve coordination problems 01:56:01 - Partial alignment 02:11:41 - AI far future 02:23:04 - Markets & other evidence 02:33:26 - Day in the life of Carl Shulman 02:47:05 - Space warfare, Malthusian long run, & other rapid fire
Full Episode: https://youtu.be/_kRg-ZP1vQc (June 2023) Transcript: https://www.dwarkeshpatel.com/p/carl-shulman Apple Podcasts: https://bit.ly/3P9rPpJ Spotify: https://bit.ly/42Vnbzb Follow me on Twitter: https://twitter.com/dwarkesh_sp Carl's blog: http://reflectivedisequilibrium.blogspot.com/
Full Episode: https://youtu.be/_kRg-ZP1vQc (June 2023) Transcript: https://www.dwarkeshpatel.com/p/carl-shulman Apple Podcasts: https://bit.ly/3P9rPpJ Spotify: https://bit.ly/42Vnbzb Follow me on Twitter: https://twitter.com/dwarkesh_sp Carl's blog: http://reflectivedisequilibrium.blogspot.com/