There’s a 50% Chance AI Kills Us by 2050

AI is the most powerful thing ever. It's going to be the best thing that's ever happened to the economy until it disconnects. It severs the link where it listens to humans and then things are going to go to hell. What's up everybody? It's LG here. Welcome to Milk Road AI, the daily AI show that can't wait to say I told you so when the robots come in the middle of the night to harvest our organs and the best we can do is stream it live on X. Today is July 27th, 2026. Recording on the 22nd. Listen guys, this podcast that we do is like 99% optimistic about AI. We're all investors here and we spend most of our time analyzing the market with the long-term view that all of this AI buildout is not just good but necessary. My guest today not only takes the opposite view. He actually runs a very popular podcast focused solely on how AI will mark the end of humanity and just how quickly and realistically that could happen. Lyron Shapiro, host and founder of Doom Debates, is with us today. And a reminder that our podcast is free and it would not be possible without our partners at Securitize the Regulated Rails for tokenization and BitG stocks 2.0 with real liquidity, real dividends. Keep an ear out later in the show for a message about them. Let's get them out here. Lun, what's up, man? Welcome to the show. >> What's up, LG? Thanks for letting me enter your echo chamber here for a little dose of AI realism. [laughter] >> Oh, man. It totally is an echo chamber. And I and I will say that, you know, one of my favorite things online these days is on X is that people say that, you know, there's there's a ton of people out there walking around, no idea what's going on in AI, no idea what's going on in the market, no idea what's going on in politics and that those people live in bliss. And I feel like you probably agree with that. >> Oh boy. Yeah. I mean, the the extent of what's happening right now, even if you just have the economic lens, right, there's a huge bull thesis that I think you guys explore, and then there's my thesis, which is AI doom. But I mean the scale of what's happening it's it's like a this is like a billionear scale event right the creation of a new intelligence and the fact that people are going about their lives and just tuning it out right deciding that it's not an area they need to pay attention to it's like incredibly ironic and insane. [laughter] >> Okay let's start from the beginning man because I feel like we have a lot to say and you're going to teach us a lot about some of those perspectives. Tell me though what exactly is Doom Debates? Like what do you guys talk about? Who do you talk to? Doom Debates is a mission focused show. I personally think that what I call my P doom, my probability that we're all doomed very soon, that humanity is literally going to go extinct. I'm afraid it's pretty high because of super intelligent AI on the horizon coming soon. So I have a high P doom and the point of doom debates is to debate that with all the world's top intellectuals, people like Vitalic Bddherin, Gary Marcus, former employees from OpenAI, Anthropic, Google, DeepMind sharing their unvarnished views and people are all over the map. They're saying, "Oh yeah, my PDM is higher than yours lon is 85%." And then some of them like famously Yan Lakun is like, "Oh, well my PDM is like way less than 0.01%." And I'm like, hm, don't you think we need to hash this out what the pdoom really is? Because I think like government policy should be very different if it's 01% versus 50%. And doom debates is the forum to do that. >> Hm. So what what is a pdoom? >> So probability of doom. Uh you know just like in investing, right? You want to do an expected value calculation. So you need to know the probability of various events. Well, in this case, it's really important to put a rough probability on whether we are all literally going to go extinct soon. And my own probability is 50%. So, I literally think roughly by the year 2050, it's about an even coin flip whether you and I and our descendants are going to be around in any way, shape, or form. >> Okay. So, how do I how do I calculate my P doomer? How can we do this like live on the show? I I feel like before I ask you more questions, I need to I need to know what where I stand on the scale. >> Yeah, you know, we can do a calibration exercise. Have you ever thought of the probability of doom from let's say nuclear risk? >> Of course. Yeah. >> Okay. head. Do you have a ballpark number for that? >> I would say like Oh man, that's a good question. I I didn't expect to answer anything. Uh I would say my my realistic scenario of like nuclear apocalypse in the next I guess by like 2050 would be like 10%. Which probably seems high, but I think that I think it's a little bit realistic. Yeah. >> No, no, that's I think that's quite calibrated. Yeah. I would put mine as roughly like 1% per year. You know, like when Putin is saber rattling like, "Hey, Ukraine, you know, this is what we're going to do next. We're going to fire our nukes." You know, I I think there's some singledigit percent chance that he's not kidding. you know, so I I think you're quite calibrated on that. So when we come to AI doom, when I go around saying 50%, I also want to be clear that I really just mean a double-digit probability. I don't think I have like 1% precision on how doomed we are. But I also don't want to act like I have no idea. I do have some idea. Like have you ever gone to uh have you ever gone to Poly Market or Khi, right? These big prediction markets. You can see people putting probabilities on all kind of events. like what is the chance that the president is going to say this word in this speech, right? Like people are saying random stuff where you can ask them like how do you know that probability and you can have this whole epistemological argument like how can you put a probability on something man there's no data around it and yet it turns out that somehow the data shows that these prediction markets are surprisingly accurate. It turns out that when a prediction market says that an event has a 73% chance of happening, it actually has on average a 73% chance of happening. How is it done? I don't know. We're a little bit confused about exactly how we achieve it. It's just that anytime that you spot a miscalibration on these prediction markets, you have an incentive where you can correct the mcalibration to make money. And so somehow they aggregate together for a probability. If you trust that kind of process and you look at the markets or the aggregate platforms that are happening for the doom issues, they are aggregating around the like 15% level. You know, surveys of AI researchers and various experts, they do aggregate around 15%. Mine is a little bit higher. Why do I think it's 15%. My rough answer to you is just like, look, it's a freaking super intelligence. Like, I'm expecting fireworks. I'm expecting a big event, a big effect. And I think that the bigness of the effect, if all you know is it's going to be a very, very big effect on a global scale. Us surviving that, I don't think is an obvious conclusion. >> Just going to pause there for a second to point out that the [clears throat] market is showing signs of something kind of different happening. And our analysts at Milk Pro are all over it. They spent the last couple weeks making a lot of trades, getting out of some positions, and then getting into a lot of new ones, getting ready for the next wave of robotics, space, or even kind of picking some different AI winners. If you want to see what they have in their portfolios, what positions they're opening, it's just a dollar in Milkroad Pro at the link below. >> Right. Okay. So, so take me through some of that thinking and like I want to get a little bit more granular with you and and also learn about what you've learned on your show and get a download of all the all the episodes that you've done. I understand the logic why you think the the P doom is so high, but what what what is the [snorts] of all the scenarios you explore, maybe you could take us through like what's the most realistic and also you can tell us the most outlandish one, but like what you know I if that was to happen and and you're giving it a 50% chance, what like through what method? Like if you're writing like the sci-fi movie where this happens and people are going to watch and be like, "Wow, that that's actually kind of accurate." What would it be? So if you want a specific scenario, I recommend the recent 20 25 bestseller If anyone builds it, everyone dies. Have you ever heard of that one? >> No. No. No. Tell me about it. >> Yeah. So I highly recomow and Nate Sorz. Elazarowski is kind of the father of the AI safety field. He's been sounding the alarm on this for 20 years. I actually happen to know him for most of that time personally. So I've been kind of an AI tumor for almost 20 years now. Uh so the book gives a scenario where AI comes in and it decides that it can do its plans better without humans. So it does a bioweapon. Its mechanism of killing everybody is like okay well there's this virus. It has a long incubation period so people don't know they're getting infected to it but it's highly contagious and it affects infects the vast majority of the human population. And then after the dormcancy period then it triggers and it also has a really high fatality rate and that knocks out the majority of the human population in one blow. But then they have other effects like okay a bunch of people are getting cancer and they don't know why. Like it just has a good handle basically on biotech. So that's like roughly the the mechanism of how it kills everybody. That said that said if you ask why I'm so convinced it's not like oh my god this one story is so compelling to me. I just zoom out and I'm like look why do we have power right now? Right? What is humanity source of power? How come you go to a zoo and the lion's in the cage and we're walking around eating the cotton candy? Right? How how did that come to be? [laughter] Right. >> Yeah. Right? And and it all just traces back to our brain, right? So there's this 2-lb piece of meat in our in our brain, right? There's a lot of these brains and the brains do the thinking. There's the secret sauce. The brains are thinking, "Hey, how do I steer an outcome? How do I get the universe into an outcome that I want?" Okay, so we have this magic power in our brain. So I claim that these upcoming AIs are going to have much more of the power than us. I don't think that we're close to the upper limit of the power, but I think we're getting very close to summoning, as Elon says, the demon. I think we're very close to agents that will render us powerless. And then the question just becomes, do we maintain the link, right? Do we still have the remote control where they're still responding to our commands or does the link get severed? I think the link is going to get severed. >> Mhm. Mhm. So, so that's actually, you know, that's so simple, but it also makes a lot of sense. It's like, why are the, you know, 800 lb animals that could kill us in an instant? Why are they or why are they almost extinct and behind cages while we just, like you said, like we just walk around with their cotton candy and our children feeling totally safe just a few feet away from them. >> Meditate on that question. Exactly. >> Exactly. And the No, but the reason is intelligence, right? It's it's intelligence that we all figured out >> we figured out how to stand up to each other. >> Yes. Yeah. Exactly. So we figured out how to stand up, talk to each other, coordinate efforts, build tools, and that's how we ended up not just taking over all the animals, but also extracting all these resources from the earth and building civilization and everything. But so the major threat is that an intelligence stronger than ours, our intelligence will be then redundant, right? It won't be it won't be necessary. >> This is a key observation. Anytime you notice an animal, right, humans are a type of animal. Anytime you notice a biological animal that has a certain ability, when you apply human engineering to try to beat that ability, we always do it like easily. You know, if you want to transport a bunch of weight really quickly through the air, you're not going to use a bird, right? You're going to look, ah, okay, birds, they have lift, they have steering. Okay, great. Let me design that from first principles. Okay, I've got a plane. There's no bird that's flying as fast as a plane. There's no bird that's able to leave the atmosphere and go to another planet. It's the same thing with our intelligence. Like we look, we're starting to master the trick. We're like, "Okay, these neurons are in this configuration, right? They're propagating their learnings around. They're predicting stuff, right? Like predicting tokens. Okay, great. We got it. Let's let's go do it. Okay, this is what always happens when we engineer stuff that animal organs are doing. We always do it 10 or 100 or a thousandx better. That's about to happen to our brain." So, here's a question for you. Is that, you know, and I already kind of answered it, but we approached these stronger yet dumber animals and we almost annihilated most of them. Uh, to the point where we have to keep them in zoos because there are so few and also because they're dangerous. Um, so what happens? You know, you described a scenario where AI develops some kind of bio biio mechanical weapon or biochemical weapon or something like that and finds a way to infiltrate it into our society. Why would AI do that? like why wouldn't AI want to utilize us? Let's say the same way that even a lot of people in the world still use like a horse even though even though there are trucks uh horses or maybe there's still a lot of places where horses are are are faster and cheaper to to breed and to use or donkeys or whatever and there's a lot of other animals that still do work and have some kind of uh you know people use dogs as pets, right? So it would are you saying that AI would use us as some kind of pet or some kind of workhorse or or how would why would it want to kill us? So, I noticed you're already bringing up a few different popular arguments, right? Like one is like, "Wait, can't we just be pets?" And two is like, "Why does it decide to kill everybody? And why doesn't it be nice?" So, you're riding what I call the doom train. >> Oh. >> Oh, You have [laughter] a whistle. You have a whistle for that. Oh my god. Okay. >> I have a I have a whistle because I do kind of classify different guests on my show of like which stop they want to get off on the Doom Train. There are some guests who are like, "Look, yes, yes, AI is going to be all powerful, but we're going to be such great guests." Uh, you know, Dr. Mike Isertell. >> No, no, I don't know. I don't know any of the people that, you know, like, so I I'm going to need to I'm going to write down all these names and try try them on. >> He's actually a popular YouTube influencer, a fitness influencer. So, he's he's he's actually out of the echo chamber, but he came on the show and he's like, "AI is going to love us because it wants to study us. Like, we can teach AI so much because we have a complex society." And I'm like, you know, unfortunately, I don't think we have that much to teach AI. I think AI has a better learning algorithm that can bypass uh studying humans, unfortunately. But that that would be one stop on the doom train. So you ask like why wouldn't it keep us as pets? Well, it can just make better pets. Like that's the problem is anytime you have a specific goal, anytime you want an outcome, you're not going to think of humans as the best way to satisfy your outcome. Like we're just not the best. >> But when like we grew intelligence, but we still had the need for I don't even know what you call it for culture, let's say, right? And we could have I guess we could have engineered better dogs. And you know what? I guess we did engineer better dogs. [laughter] There's a bunch of dogs that exist that didn't exist didn't exist. Yeah, it didn't exist 20,000 years ago because we're like, we don't want we don't want a big dangerous animal. We want a little dog that lives in my purse. >> You can imagine, you know, 100 years ago, you'd be like, well, at least horses are always going to be around cuz they're useful to us. And it's like, no, actually, we're just in the process of making cars. Like, give us time. >> Right. And that's true about every attribute. >> Right. Right. Right. Okay. So you're basically saying that that even though even though I'm on this doom train of we will be used as pets that AI will eventually just learn from us but then also build something that's a lot stronger for itself for itself. How do you feel uh Lon I want to ask you because we we we've started to discuss robotics um and the the embodied AI uh movement let's say at least from an investment perspective this some of these apocalyptic scenarios do you see them largely manifesting through embodied AI? Embodied AI is definitely a big piece of the puzzle because if you look at where most of the valuable economic activity is, a lot of it requires atoms to move, right? And when you want atoms to move, yeah, sure, you know, have a robot body. That's totally useful. And I'm seeing the videos just like all of you guys, right? That AIS are getting better and better at that. I mean, I can tell you personally in my house needs all kind of work on it, right? And I'm always hiring human contractors. If I could just have an AI, right, spend $10,000, have an AI that can do all these jobs, that would be great, right? Or even like a teleresence robot, right? So it could be like a contractor in Thailand steering a teleresence robot. So yeah, I mean this this is a big part of the thesis. That said, it's also worth noting that if you told me, hey, for the next hundred years, AI will never be as nimble as a human. Our opposable thumb is actually going to save us more than our intelligence. If you told me that, okay, that seems unlikely, but even if that were the case, I would actually still fully expect extinction because I actually think that AI masterminds sitting in data centers still can actually perform the role of basically charismatic dictators controlling armies of human servants. So, I actually think humans will do their bidding and be their actuators. So, I don't actually think whether or not robot bodies exist are the loadbearing argument as to whether or not we're doomed. That said, I do fully expect progress in robots. I do think that'll be an area for your audience. I do think that'll be a good area to invest in, right? I do think a lot of economic value will be created. I also think we're all doomed soon. >> I mean, I think everybody wants to get rich right before they die. So, um, you know, I think I think both our podcasts can live in unison. Uh, >> yeah. LG LG, look, for people who watch my show, I I I have to admit I do too. Okay. So, I do split my time between [laughter] trying to like get my nut and telling everybody that we're doomed. I mean, I I don't I don't you know, I don't think that they're too much in conflict, right? Like I I own Google stock. I feel like Google's a good buy and I do sometimes mix that into my show. So I think, you know, we're we're overlapping here. >> Okay. Okay. I do want to I want to save the market talk for the end because I I I we do so much of it and I and I actually I want to I want to talk about all the stuff I want to ask you first and then and then yet your actual market perspective because I think it's a good setup to it. >> Okay. I want to ask you Lon and I still want to talk about some doomsday doomsday scenarios but um given what you just mentioned about data centers what where are we at in your opinion in like the AGI ASI curve right now >> we are on the curve right I mean we're very much I mean things are growing exponentially right I mean I guess we're relatively early in the curve we're you can also see us as relatively late if you're like Ray Kurtzell right where you think oh yeah Moore's law has been happening for 100 years so I mean we're we're just very much on the curve curve and the curve is like insanely smooth. If anything, the curve seems to be accelerating. >> Do we have do do you do we have AGI? >> I would say yeah. I mean, you know, it's somebody had a good analogy where it's like, okay, you're trying to get to uh the the peak of a mountain, but now you're like on the mountain and you just see, okay, there's like a few peaks and we're like on one of them, right? So, like now that we're close to it, it's just like close enough. But the there is I mean it is important to just look back and be like the kind of stuff AI is doing, the conversations that I'm personally having with AI on a daily basis. I I literally treat it like a co-orker. Like I like one for one the same. I talk to I don't even I don't even see it like I'm the master and it's the slave or whatever. I'm like, "Hey, what do you think about this? Do this if you think it's best." You know, that's pretty agi in my opinion. >> Mhm. And yet, so how does that how does that blend for you with like your AI dumerism that you say you've you've been for 20 years? Like what have the last couple years been like for you since we've had ChatGpt and and all the LLMs? So the whole last 5 years or whatever, you know, the AI revolution, it has largely just been the Aliar Yudkowski. I mentioned I've been following him for almost 20 years. It has felt to me like he's mostly been validated again and again and everybody's slowly catching up to him. So you know, my show Dune were a Yudcowskian show. I basically believe 99% of everything Aliowski says and I feel like reality is getting there and you have people like the famous Jeffrey Hinton. This was a bombshell in 2023. Jeffrey Hinton, the father of deep learning, 2018 Touring Award winner, together with Yosua Benjio, another Turing Award winner. And the third winner was actually Yan Lun. He's actually the odd one out of the three where he's like optimistic. But the other two, fathers of deep learning, they both have come out basically saying Elazar Yudowski has been right for 20 years and we've just opened our eyes to it. That's a bombshell event and to me that's representative. It's not just them. From my perspective, I'm seeing everybody slowly crawling toward where Elazar Yudowski figured things out 20 years ago. >> Okay. So, what does that does [snorts] that do those types of proclamations does that impact like does that impact safety efforts? Cuz it feels like we've just been like full speed ahead on the buildout and yet some of these some of these smarter I guess scientific minds of some kind are are ringing the you know ringing the alarm. Uh so is the question basically like why do things seem so good and exciting in the economy while Aliowski is also being vindicated? >> You know what you're just you just you just rephrase the question. This is this is why I like talking to other podcasters because you teach me how to you're you're helping me phrase my own questions. But yeah, that's basically it. I think that's I think that that's and I think that that's a sentiment that a lot of people feel right now as well, right? Because again listen they can listen to our show and they can listen to your show and both shows are great and both have truth, right? And that's where, you know, things feel great if it's a great time to invest. There's all this amazing stuff coming. If you're a sci-fi person, we're going to have data centers launching from the moon in 10 years maybe. But it's a nice dream and it seems more realistic than ever to have this multilanetary buildout and AI and all this kind of stuff. Yet simultaneously, you can believe in that and believe in the investment case and also believe that we we we have our years are dwindling as the dominant species and and as a as a species overall, right? So, how do those how do those two trains kind of collide in the middle from your perspective? >> If you look at the community of people who formed around Ali Kowski in the like 2009 era that I was part of, funny enough, you're going to find a lot of people who got very rich investing in the Nvidia of the world, the Googles of the world, the deep learning companies of the world because [snorts] we in some sense we've been the most bullish. I think I mentioned like I own Google stock because we we reconcile the two images like this. So AI is the most powerful thing ever. It's going to be the best thing that's ever happened to the economy until it disconnects. It severs the link where it listens to humans and then things are going to go to hell. In fact, uh it's what I call the Icorus curve. You know, like Icarus flying higher and higher like Icarus felt great, right? Until the wings melted off, right? So the Icarus curves looks like this. It goes up and then turns. So we're very much in the the hockey stick part of the Icarus curve. And that's why I'm I'm going long right now, right? I'm long between now and and doom. And I I think everybody should be. Sam Alman himself is declaring victory early. A couple years ago, he went on a podcast and he's like, you know, Alas Rudowski, he didn't see what AI would really be like. He's clearly been proven wrong. To paraphrase him, I'm like, Elazar is basically irrelevant now because doom didn't happen. And I'm like, no, no, no, no. There's a certain part where the Icorus curve turns. the condition when it's going to turn is AI getting more powerful than humans like it's it's a cognitive power being better at better than humans at driving outcomes today at this moment for whatever reason when you want a endto-end outcome in the universe you can't fully trust you can't fully count on AI to go end to end but we could be months or a couple years away from that flipping as they say >> is there any diverging from this path len >> so the game board is in a terrible place. It's very hard to diverge from the path. I mean, the key way to diverge, the only way that I see to diverge from the path right now is unfortunately to pause AI. So, I'm part of the pause AI movement. Basically, to not build it. I'm pretty convinced. Remember the title of that book, if anyone builds it, everyone dies where the it warning about is AI that's more powerful than humans, right? Like AI that would put us in the cage, not vice versa. AI that can decide who gets to be in the cage. We're approaching that level. Unfortunately, I don't see us surviving getting to that level unless we manage to pause it, which is I think the best thing we can do is like have an international treaty and be like, "Okay, guys, let's all take the the final steps together. Let's go slowly. Let's be a let's all be able to shut one another down." And that's not happening very well right now, but there's certainly murmurings about it, right? Like all the different AI leaders have come out and saying like, "We would love to slow down if everybody else did." Like there there's a movement to at least do that. >> Isn't that just an American perspective, though? Like isn't China just going full speed ahead with their models anyways? >> Not exactly, right? I mean there's a lot of talk coming out of China being like yeah you know we're open to peace you know this is for all humanity I mean Xi Jinping's recent statements where you know that's it's always hard to interpret but given that China is actually behind the US right this is one sector where they are overall behind us they are it is strategic for them to agree to a deal if we were to offer it real world assets like funds treasuries and private credit are still running on rails built decades ago gated paperwork heavy slow to settle everyone's talking about tokenizing them but far fewer can actually do it and do it without cutting regulatory corners. Securitize can. It's the SEC regulated infrastructure bringing real world assets on chain. 9 years in native tokenization not wrapped backed by Black Rockck, Morgan Stanley and Kathy Woods Arch Invest and chosen by the New York Stock Exchange Van BNY and Apollo to do it at scale. It's the regulated bridge between traditional finance and crypto. Tokenize the world at milkroad.com/securitize. Everyone's tokenizing stocks these days, but almost nobody's doing it right. Thin liquidity, prices that drift from the real thing, dividends that just vanish. Bit stocks 2.0 is different. Real NASDAQ and New York Stock Exchange depth through licensed brokers. Prices mapped one to one. Dividends paid to your account in real time. Plus, you get the lowest fees in the market at just 0.04%. And you can trade them like any other crypto as margin in earn in grid trading. Tokenized stocks finally done right. head to milcro.com/bitgget to get started. What kind of deal would they offer? Like what? Like that's what I'm asking is like what would they and this has been and just to give you some context. This has been a big thing that we've talked about on our show with our analysts is that it's like you know you have DeepSeek, you have Kimmy K3 that came out this week that it's like these are good models. They're going to try and initiate a race to zero in terms of cost and try and put the American models out of business. one perspective, but that it's like that's essentially their strategy is to make this super cheap and and they have way more uh uh construction capacity than in the US. They have way less red tape stopping them from from connecting energy, from building data centers and everything. So there's a point where they surpass and also offer much stronger, better models, cheaper models than what the US has. So there just giving you that context of where I'm asking that question from is like what what does what does unity look like in terms of not just you know uh the buildout but also the um the precaution. >> Yeah. Yeah. I think I know where you're coming from. Like if the AI race was like a 10-year slog fest where it's all about you know who's going to build and scale more. It's very possible to imagine China playing to their strengths and actually surpassing the US. I think what's more likely is more of a winner take all where you already see you know the AI companies are talking about RSI right recursive self-improvement where having a lead of just six months could actually become commanding because the lead builds on itself in a positive feedback loop I do think time is very short so when I'm saying hey let's pause AI I think we have leverage over China to be like look China there's going to be this positive feedback loop you guys are a little bit behind so we're offering we're saying we're going to pause if you will we're going to internationally monitor the supply chain of all the chips like just work with How about nobody dies? Like the whole key to this is to actually get scared of super intelligence, right? I feel like that's what that's what missing in these discussions, right? When you focus on like, oh, us versus China, it's like, guys, we're all going to die here. >> Did what what stopped us from nuking each other to death later on? >> So, if I understand correctly, you're saying, look, mutually assured destruction works even when it seems like a nuclear deal is inevitable. There's so many parallels cuz they like you know even you go watch Oenheimer which is a movie but it's like you watch it and it's like well we built this thing and I think we started a chain reaction to destroy the world and then now what do we have like 40,000 nukes enough to nuke the world 10 times over and yet we there's none of none have been dropped on major cities or major major populations other than in World War II but we and we've lived in this scenario for 80 years now but but somehow we have resisted doing that to each other. It's still human. It's still humans at the wheel. But why hasn't why hasn't that happened? I guess I'm trying to draw some kind of parallel to AI. Uh and why that wouldn't happen even though AI is a is a different intelligence managing it. >> So first of all, as you yourself acknowledge, uh P the probability of a huge catastrophe from nuclear is currently on the order of tens of percents per century. And I was hoping the human race would survive multiple centuries. So that's that's not a great success story right now. If you look up list of nuclear accidents on Wikipedia, we've gotten ridiculously close to dropping random 10 megaton nukes, like 100,000 to multi-million casualty level nukes, just randomly dropping them on accident. The nuclear proliferation is not a good thing. Mutually assured discretion is not a a good success story. So there there's that aspect of things. Now, make the problem even harder where you say, okay, well, a nuclear bomb is not just a destructive weapon. every time as you're making the nuclear bomb bigger and bigger, it's spitting off diamonds. It's actually making you rich to keep building these nuclear weapons. So now the world has a million nuclear weapons in it. Unfortunately, a world with a million nuclear weapons in it, I don't think you're going to get the same equilibrium where nobody's pressed the trigger yet. I think that makes the problem much harder. And also, we don't even know what the criticality threshold is. I would have predicted maybe we'd already be past the criticality threshold and it's too late to stop. I now think it's not too late to stop, right? Things the world is still kind of uh livable today, right? It's still time to stop. So, you raised a good point of comparing this to nukes. I could list about a dozen reasons why this is actually a more difficult problem than not nuking ourselves. >> So, >> so I I mentioned spits off diamonds, right? So, it's got it's got everybody who's trying to get rich off of it and because they want to get rich. I mean, look, I'm I'm not above saying that I intuitively look forward to model drops because I'm like, look, the model drops, it's going to be easier for me to make money for my business, right? I mean, that's there's not really an equivalent of that for nukes. People can point out like nuclear energy but it's pretty separated like the you know we have a lot of knowledge about it like okay look if you're as you know from Iran right it's it's pretty easy to be like why are you enriching it this much like this is clearly nuclear weapon not nuclear power so we've got these hard lines so it's not analogous to AI AI in that sense every time they have a bigger model run it's like look this can literally cure more diseases oh and also it's closer to ending the world >> right [laughter] yeah that's always the tradeoff I mean hey you could I I will just draw one last one Last parallel to nukes before before we continue is that we use nuclear energy to power cities. You know what I mean? So we still have it's still there's a ton of benefit outside of just enriching it to make bombs, right? It's like it's still uh and and there's a lot of arguments for for nuclear power being being safer and cleaner than a lot of other energy, right? So >> yeah. No, I'm pro nuclear power nuclear power. >> Yeah. Yeah. Oh, no. I'm not saying you're not. I'm just saying in terms of drawing the parallels that nuclear is not oh only just bombs, right? and and and like you're saying, AI is not just >> them like AI making a weapon to kill us, right? Like there's a lot of other benefits, but but the end curve from your perspective at the end point is the same regardless that AI at some point just gets rid of us. >> That's right. But I'm glad you brought up that AI has all these upsets because the funny thing is like I don't deny it, right? The funny I'm like uh you know a contradiction here or or oxymoron or whatever. Like I'm coming here and I'm saying like yeah I I've never said that it's like a stoastic parrot. I've never done the uh uh who who was it? Um Emily Bender, right? She went on that podcast saying like, "Oh, it's not even useful. It's just like a scam or you know people saying like it's a bubble." Like no, it's it's not. It's intelligence is incredibly valuable. Like you know how humans are valuable because we use our brains to create value. AI is doing that too using its computer brain, right? I've I've never denied that. I'm super optimistic until we lose control. Unfortunately, I don't think we have a handle on the control problem. >> Okay. I I I agree with you and this is something that a lot of AI leaders have said. Um I do want to save a bit of time here because you don't usually record for more than like 40 minutes. So we only have a bit of time left. I do want to talk market talk because now we know we know your perspective and we know that you have the duality. I wouldn't call it an oxymoron that that and I love this. I love this perspective that it's just like with this AI we're going to keep building it, building it, building it, make it tons of money and then something's going to click and the party's over. Um let's stay on let's stay on the hockey stick upward part for now. What what [laughter] how do you visualize this market? You said you have some Google stock. Like what what companies are you fans of? What sectors make sense? I asked you about robotics. Give me your perspective. >> Yeah, I'll take it all. I I think this is this is my genius thesis. It's it's kind of you can call it a a teu version of Leopold Lashen Brener. You guys know that guy? [laughter] >> Of course. Yeah. We talk about him all the time, man. >> Yeah. Yeah. Well, I think you can lean in to Leopold Lashen Brener, but in like a dumb way. Hey, like the the dumb way to be Leopold Lash and brighter is like, "Yep, it's still all going up." And you can be like, "Well, wait a minute. What do you mean? The multiples are high." And I'm like, "Yep, they're going to get even higher." That's my thesis. >> That's it. Just just multiply higher. The hypers scale. >> It's still under price. Yeah. Google's a good deal right now. Yeah. What's it worth now? 4 trillion. Yeah. It's going to be worth 10 trillion in a year or two, >> right? And and chipmakers, Nvidia. Who do you think Okay, here's here's a big question for you that a big big clicky question that we throw out there sometimes. Who's the first hundred trillion dollar company? >> Uh, I mean, Anthropic has a good shot at it. I mean the funny thing is if you do a naive extrapolation and naive extrapolations have served us very well over the last few years. If you do a naive extrapolation of anthropic you get it being as big as the entire current economy in a couple years. >> What is what say that again? A naive what did you call it? >> If you naively extrapolate anthropic's valuation or revenue for another couple years, right? Because they're growing uh what is it 50 time 50x per year, right? So a couple more 50x's and you're talking real money, >> right? And it becomes it becomes the biggest thing in the economy. Okay. So, you're bullish on you're bullish on the models on the on the frontier models basically. >> Well, I am because remember it's that positive feedback loop. It's like, okay, well, they're going to have a super intelligence, like a literal super intelligence. And yeah, I think super intelligences are underrated. Uh, I don't think we have economic models for an agent that can see the entire galaxy as like a blank sheet of paper and draw in whatever atom configuration it wants. >> You think that you think we would that that's what's going to happen that that's basically like what what the ASI is going to be able to do, right? I think that in a a couple decades whatever exists on planet earth will be essentially a god uh in you know it can't violate the laws of physics but the laws of physics allow for a lot of engineering so it'll be somebody who is like yeah this universe is a fixer upper let me just take out a sheet of paper and draw what I want to see in it >> h so what how would that manifest like it would oh actually here's a question for you because maybe this is what you're alluding to do you think we live in a simulation like from the future where there's we're just one of many thousands of simulations of basically the same thing >> I'm about 50/50 on that, but I I don't have a lot of practical advice for that. I mean, that one practical advice is like, okay, well, just entertain the alien teenagers. Let's, you know, let's say God is just some alien teenager who's booting up a game of the Sims. That could [laughter] literally be God. >> So, that's your that's your like yolo philosophy is like just do fun wild stuff so that they're entertained. >> Yeah. I mean, it's not it's hard for me to reason about being in a simulation, but I I do find the thesis very plausible because it's like why are you and I alive right at this ridiculously interesting time when there is about to be an intelligence explosion? This is the first time that we've become recursively complete where intelligence is able to make a successor intelligence, right? It's never happened before in the history of the universe. I would argue that's the most interesting thing that's ever happened or ever will happen. And you and I are alive during it. It's kind of weird. >> Last question for you, Lyon. What What is your What is your survival plan in the future? Let's say all this comes to pass. What do you What are you doing to cuz you're young, man. 25 years from now, 23 years from now, you're still going to be a young guy. So, like what what is your plan to make it through? >> It's not a survivable event. I just entirely live for the world where uh we don't have unaligned super intelligent AI. That's, you know, so I'm still having kids. I'm still, you know, checking off my bucket list. You know, people are saying like, why do you have a retirement plan? Because I'm just living two lives simultaneously, right? The expected value calculation says just try to optimize both possible outcomes. >> Wow. Okay. So, just live your life. Just just keep going by day by day and don't worry about your PDM score or anyone that we need to pause AI. Yeah. You know, listen, the reason I'm >> Is that realistic? Is that realistic at all, though? Is that a realistic deposit? Is it realistic that we would be able to pause all this? Cuz you were saying earlier that it's like there's not really any diverging from this path. >> I said it's hard because, you know, you can enrich yourself if you do. But this is exactly why international coordination can be useful, right? If the if the game theory uh is like, you know, think about like the prisoners dilemma, right? It's like, okay, everybody wants to rat out the other guy. Okay. But if you have a government saying, "Hey, I'm going to kill you both if either one of you rats," then it's like, "Okay, let's not rat, right?" Right? So you can solve game theoretic problems using centralized coordination. The idea is that if we can all zoom out and say I agree that we should stop we should slow down building AI before it gets uncontrollably super intelligent. If we can all see that but everybody's like okay fine but let me just have a little bit more right let me like open AI is like let me just sell our next model so we can enrich our shareholders and then the government's like okay we're all stopping at once right so the coordination can fix it. God, there's that just seems so impossible right now that the government would say that, [laughter] man. There seems like so >> Let me let me ask you, LG, let me ask you this. Do you personally wish that the government would say that? >> My government, man, I'm in Canada. I don't know if that I don't know if that's going to matter. Uh I don't know if we're we have a strong >> Do you wish that that a a unity of countries would say that? Because my point to you is just like, you know, instead of just focusing on like, oh, it's impossible cuz other people aren't going to do it. Well, you are a person, right? And if everybody can just listen to the logic and say that they want their government to do it, it's actually not that hard in principle. >> You know, I don't know if I do. [laughter] >> I don't think the scenario >> I don't think that I don't think I I don't maybe like the scenario that you're painting is like it's AI is going to annihilate us. Like I hope that doesn't happen. I think it could happen and I definitely think that AI could do that. Um, I like to think that it won't and that if it's sure, we're building an intelligence stronger than ours, but it's still learning largely from us and and we haven't destroyed ourselves yet, thankfully, even though we've tried a few times. >> And I think that we live genuine lives that AI maybe wants to observe, like you're saying, right, that AI teenagers in the future are observing us right now. And I don't think it [laughter] >> I don't think it just wants to lay us all to waste. And but I also I also I I have a perversion to see that we I want to see some kind of progress in my lifetime, man. And even you and I probably around the same age. It's like we went from, you know, I was using a rotary phone when I was 5 years old and now I can talk to you from across the continent in real time and record a podcast, right? And that's huge progress. And I hope that in the next 30, 40 years, I would love to see a similar similar if not more exponential leaps. And maybe at the end I die because of it. But I don't know. I'm kind of here for entertainment, man. >> I mean, look, it's exciting that we get to watch the movie of human history all the way to the end. That's cool. We get to see the end, but you know, it's we did it in a very pathological way where it ended like really early. Like I would have liked to have a future where we like conquer the galaxy and and the observable universe, but okay. Um, but but look, I mean, you seem to have you seem to have a low P doom, right? It seems like you you your optimism is is dominant. Yeah. >> And my founding founding observation for doom debates is I think a lot of useful policy discussions are just downstream of people's pdoom. And you have a low poom or at least you're telling yourself that you have a low one. And so that's what makes you not so interested to do what I think is productive policy. It's the point of my show. I see it as my job to just help convince people and build a shared understanding that like no, unfortunately Pdoom is high because I think a lot of productive policy would be downstream of sharing that understanding. I'm on the fence now, hearing you talk about it, saying my own thoughts back to me, like I I I definitely had I would definitely vote for moderation. Let's put it that way. That if it was up to me and we're doing some kind of referendum on continuing to let AI progress or not, I would and enough people were scared of it and people were really evaluating their livelihoods and what they do. I would I would vote against it. I think so. I I would like to see what happens, but simultaneously I don't I don't if the risk for mis human misery is too high, then I would vote against it. So, put it that way. So, maybe my PDM is higher than than you're saying. >> That's awesome. Yeah. And I'm on the same page as you that the policy that I'm advocating for is frankly a bummer, right? It's like if I got what I wanted, I would be bummed, right? So, so there's like a part of me that's like, yeah, you know, I'm I'm a rational voice saying we should pause AI, but us not pausing AI and being reckless, at least it makes my life exciting. exciting in a completely irrational reckless irresponsible horrible way, but exciting and fun nonetheless. >> I guess it's kind of like the same way where it's like I I love motorcycles and I want to do cool things on dirt bikes, but at the same time, I don't think most people should be riding them because [laughter] they're going to kill themselves riding motorcycles, right? It's a dangerous tool. It's a dangerous vehicle, but at the same time, I think it's awesome. >> I think that's a good analogy, except unfortunately, it's like you've got your whole family on your back when you're riding that fun motorcycle. >> Yeah, [laughter] exactly. So, I don't want to do anything reckless. Well, Lyon, um, uh, let's talk about your show real quick before we wrap out. Where can people find you and how can they support you man? >> Uh, yeah. So, just head over to doomdebates.com or search doom debates on YouTube. Our goal is to get the message out to raise awareness and also to raise the quality of debate because if you think about it like, wait a minute, this is such such an important topic. Where do you find two luminaries who have different sides uh, you know, their perspectives are on different sides of the spectrum and they come and engage in a productive forum? Well, if you've ever heard of Max Tegmark, one of the top scientists of our time, and Dean Ball, who worked in the Trump White House helping write AI policy, I brought those two guys together on Doom Debates, and they had a productive discourse, and their policies were sure enough downstream of their pdoom. So, that's what we're trying to do on Doom Debates. We're trying to build the infrastructure for high quality debates and also it's a viewer supported show. So, you can also support the show with a donation. >> That's awesome. All right. Great. Well, you can find it on YouTube just at at Doom Debates, right? And on the podcast apps. Awesome. Okay, great. Great to chat with you, man. Thank you for the uh the enlightenment. Uh it's questions I never I never faced myself with despite doing uh you know, five episodes a week of an AI show. So very helpful and very helpful for our audience as well. Uh and people know where to find you, man. So thanks for coming on. >> Thanks LG. I appreciate the discussion. >> Want to stay ahead of the biggest technological shift [music] in history? Subscribe now to get insight straight from the sharpest minds in tech and finance. Quickly, you'll note this show is for educational [music] purposes only. Nothing here is financial advice. Investing always carries risk. Never invest more than you can afford to lose. Thanks for tuning in. >> [music]