97% Invincible

Aaron Stein 0:02

Because you want to put billions of dollars for a joke.

Jeffrey Lewis 0:08

You are listening to the Arms Control Wonk podcast, the leading podcast on arms control, disarmament, and nonproliferation. I'm Jeffrey Lewis, a scholar at the Middlebury Institute of International Studies at Monterey and the founding publisher of the Arms Control Wonk blog.

Aaron 0:23

And I'm Aaron Stein, a senior fellow at the Atlantic Council's Rafiq Hariri Center for the Middle East. I had to read that four times, Aaron. Well, you just let the listeners know that we aren't perfect every single time, even though we mostly tape live. We tape the pod live to tape. We have three junior fellows running around the house.

You may hear them. That's right. You are in upstate New York, still on your East Coast swing, and you are on dad duty. That's right. I am sitting here in New York.

Jeffrey 0:53

Protected by the amazing missile defense system that Donald Trump assures me has a 100% chance of shooting down a North Korean missile if only we fire two interceptors. Was it 100% or 97%? Oh, it was 97% for each interceptor. And if so, if you fire two, that's 100%.

Aaron 1:23

I did not do well at math in school, but even I know that there is some sort of arithmetic challenge going on in that one.

Jeffrey 1:32

We thought today we would talk about claims of missile defense effectiveness because it seems that I think there may be some misunderstandings out there about just exactly what it is that the ground-based midcourse defense system sitting in Alaska can do against North Korean missiles. And so I thought we would have a conversation about that. There have been some really nice pieces.

There was a Washington Post fact check that had an extended quotation from our friend James Acton. And Joshua Pollack wrote a nice piece for Defense One. And so I thought this would be a good conversation to have.

Aaron 2:06

Yeah, and I think Joshua Pollack is also a friend of our sponsor. Yes, he is.

Jeffrey 2:11

This episode is brought to you by the Cato Institute, which invites you to join Josh and other speakers in a half-day conference, What Are We Going to Do About North Korea?, being held on Monday, November 6th at the Cato Institute in Washington, D.C. With the North Korean crisis now one of America's top foreign policy priorities and no immediate answer in sight, this free conference, including a keynote address by former governor and North Korean negotiator Bill Richardson, brings together experts who will be discussing what approaches should be taken, the state of North Korea's nuclear capabilities, how to maneuver through the escalating war of rhetoric and provocative actions, and more.

You can attend in person or watch the conference live online. For all of the details, go to Cato.org forward slash North Korea.

Scott LaFoy 2:57

And they can follow that same link to RSVP. You'll need RSVP to actually attend the conference. So head over to Cato.org forward slash North Korea to RSVP.

Aaron 3:07

Hey, they also have a new policy brief out, right? Yeah, they wanted to not only tackle the North Korea issue, they looked after Iran and all of the stuff that we talked about in episode two of podcasts ago about the decertification and the policy course that they advocate, which is the Trump administration should engage with the Iranians.

So go check it out.

Jeffrey 3:28

Yeah, that's called Unforced Error, the Risks of Confrontation with Iran by Emma Ashford and John Glazer. You know, with all... Speaking of errors. Speaking of unforced errors, that's right.

Aaron 3:43

Mathematical errors.

Jeffrey 3:44

Mathematical errors. Yeah, so there are two things we're going to talk about. One thing we'll talk about is... The claim that either the system as a whole or even the interceptors are 97% effective, which I think is fairly ludicrous. But first, we have to make fun of Donald Trump's terrible math skills. In his defense, do you remember Tommy Franks?

Oh, how can you forget Tommy Franks? Well, Tommy Franks puts the following story in his memoir. So it just goes to show you that like you know, self-awareness is a talent. He was talking about the strike on Dora Farm at the beginning of Operation Iraqi Freedom in 2003. And this is how he discusses managing the risk or the risk of failing with Buzz Mosley, who was later Air Force Chief of Staff.

What are the probabilities of mission success with a single aircraft? Franks asked. Buzz didn't hesitate. A single aircraft gives us a probability of 0.5. Again, I thought before speaking. Normally, I was a glass-half-full optimist, but tonight there could be no risk of failure. What does it take to get to 100%?

Scott 5:05

Two planes? Probability of success goes to 1.0, General.

Aaron 5:11

Oh, Tommy Franks, ladies and gentlemen. Yes, probabilities are additive, I guess.

Scott 5:21

Yeah.

Jeffrey 5:22

No, Tommy, it's 75%. So, I mean, yeah, it's the classic thing. It's like, oh, it's 97 plus 97. Oh, it's got 180. 194. Yeah, 194% chance of success.

Aaron 5:40

We'll get it. We'll get it.

Jeffrey 5:44

Probability of success goes to 1.0, General. Oh, God, it's a miracle we're not all dead.

Aaron 5:52

That's right. So Trump fell for the math trap.

Jeffrey 5:59

Yeah, Trump fell for the math trap. Now, it is worth noting that, you know, Trump probably didn't pull the 97% number out of his butt. I mean, there are a number of statements suggesting that the effectiveness of the missile defense system would be in the 90% range. And I've even seen officials say it would be in the high 90s.

There's a really nice list maintained at the Mostly Missile Defense blog by George Lewis called Updated List of Claims about GMT Effectiveness. And basically every time a U.S. official gives a number, George updates it. Interestingly, he hasn't added President Trump's claim.

Aaron 6:38

I know. I'm looking at it right now and it's not in there yet. So catch up, George.

Jeffrey 6:43

I mean, so I think I know how they get that number. But it is not the way Trump gets that number. And so we should probably talk about this because I do think that it's pretty illuminating in terms of how the missile defense system works. It is. It is. So, you know, you whip out your calculator. We should have arms control wonk branded calculators.

Aaron 7:07

Swag. That's $20 a month on the Patreon page. Yeah. Joke.

Jeffrey 7:12

And so what people are saying is, and I used to use a different set of numbers that are not so far off, but it's fine. I mean, they're approximately the same, that the Missile Defense Agency figures each of the ground-based midcourse interceptors sitting in Alaska. And there are, eventually they're going to be 40. I think they're only at 36 right now.

And then there are another four at Vandenberg Air Force Base. They're going to fire four interceptors at every North Korean missile.

Aaron 7:40

Per one missile, four per one.

Jeffrey 7:42

Right. And is that not what I said? Did I screw that up? No, you said it just fine. I just clarified. Yeah, because we're about to do math, and that will cause people's eyes to bleed or their ears to bleed. What bleeds? So there's a 60%... They stipulate that there's a 60% chance that any interceptor is going to hit, which leaves you a 40% chance of missing.

And so if you take four shots... 0.4 to the fourth power is 3%. So that's how you get 97%.

Scott 8:17

Now, I used to hear that the number was 50%.

Jeffrey 8:24

And so I used to imagine that it was five interceptors to get to 97%, but, you know, whatever. Conceptually, that makes a lot of sense. That should freak you out, of course, because if there are 40 interceptors, right, that's enough for 10 North Korean missiles. That's right. You know, there was a couple of years ago where Chuck Hagel announced that they were going to go up and they were going to add, what, I think 14 interceptors?

Aaron 8:54

Yeah.

Jeffrey 8:55

You know, and that was going to cost more than a billion dollars. Right. And so it was like, good news, 14 interceptors. As long as they don't build four missiles, we're fine. Yeah.

Aaron 9:10

Right. And the cost, the inverted cost curve, because how much does it cost them to build one missile per R-14? Yeah.

Jeffrey 9:17

Yeah, yeah, yeah. And I mean, this is why back in the 1980s when people took this stuff seriously, you know, the argument was missile defense needed to be cost effective at the margin. If the cost of four interceptors vastly exceeds the cost of an additional North Korean missile, they'll just build their way out of the problem. Right.

So you incentivize them to build their way out of the problem. That's right, because it's a good trade for them.

Aaron 9:43

Right.

Jeffrey 9:43

But there is another kind of creeping problem, which is, you know, if you think about it for a second, let's say you really do get a 97% chance of success, right? And so there's only a 3% chance of missing. You actually have to repeat that experiment 10 times. Right. Which, you know, ends up reducing your effectiveness to 74%, which is good, but you're taking a 25% chance one of those suckers gets through.

Aaron 10:24

Mm-hmm, mm-hmm. This is always the other argument against this system is that You're still, even with a system that works relatively well, you're still taking the chance, a one in four chance, that one nuke slips through.

Jeffrey 10:39

Right, so there's a 25% chance Even in this very optimistic scenario that one gets through, assuming that the North Koreans are willing to constrain their arsenal of missiles so that it's sized to our missile defense. Right, because they would be so kind. But the problem is, right, if you repeat this experiment, even if you can keep up with them, right, and so you've got a 97% chance of success, but instead of that being 10 engagements...

Let's say it was 25 engagements, then actually you're under 50-50. One is more likely to get through than to not get through.

Aaron 11:21

Right, because it's math.

Jeffrey 11:23

Yeah. Sorry, Tommy. But I mean, there is this idea, I think, that like, oh, 97%, that's great. Like, well, that's actually not all that great if their goal is only to get one through. Right. And I mean, I think if you're the North Koreans, you're totally happy as long as, you know, Little Baron and Melania perish in a thermonuclear hellfire.

Scott 11:49

So if you don't want to do this math yourself or you have trouble keeping a bunch of probabilities in your head like I do, I went over to Desmos and also built my own website for putting these calculators together so you can move a bunch of sliders and change effectiveness of the interceptors and stuff and see how that affects the probability of one getting through or the probability of none getting through.

And I think it's really helpful. I'll put them in the show notes so people can play with them if they want.

Jeffrey 12:15

Hey, Aaron. Yes? Do you know what would be a better idea than trying to shoot down all these missiles? No, tell me. How about not triggering nuclear crises with countries that are building the missiles?

Aaron 12:33

That would be wise.

Jeffrey 12:34

That would be wise. Don't provoke. Which I think, and you know who has some suggestions about how we could do that. I have an idea, but tell us. Our sponsor, the Cato Institute, which invites you to participate in a half-day conference. What are we going to do about North Korea? It's all I can do not to rename their conference.

Aaron 12:55

What the hell are we going to do about North Korea? It's only half-day, so it's a good time to go hear from good speakers.

Jeffrey 13:03

Yeah, it's going to be held on Monday, November 6th at the Cato Institute in Washington, D.C., With the North Korean crisis now one of America's top foreign policy priorities and no immediate answer in sight, this free conference, including a keynote address by former Governor and North Korean negotiator Bill Richardson, brings together experts who will be discussing what approaches should be taken, the state of North Korea's nuclear capabilities, how to maneuver through the escalating war of rhetoric and provocative actions, and more.

Attend in person or watch the conference live online. For all of the details, go to Cato.org forward slash North Korea.

Aaron 13:38

And this raises, you know, can we call it James Acton's, you know, frequent drop-ins to our podcast? You know, he raised a really good point about even challenging the math further, even with the calculator Scott just mentioned.

Jeffrey 13:52

Right, because, you know, so you then have to ask the second question, well, where does this 60% number come from, right? And I think that's an aspiration, right? That has kind of gotten baked in, but there is no evidence for this whatsoever, and actually quite a bit of evidence that the number is not 60%. So the first thing I would say is, if you look at the report of the Pentagon's director of the Office of Testing and Evaluation, No, they have this very cautious thing they say about this system, which is they say that it has a limited capability to defend the U.S. homeland from small numbers of simple intermediate range or intercontinental ballistic missile threats launched from North Korea or Iran.

But then they refuse to provide quantitative assessments of the performance, citing the lack of key testing, right? They won't put a number on it. And as you say, the problems that we have seen, and they say that the Missile Defense Agency continues to discover new failure modes during testing, those failure modes aren't independent. No. One of the problems is they had a faulty divert thruster.

Aaron 15:15

Exactly. And those are common across the missiles.

Jeffrey 15:18

Yeah. If the divert thruster doesn't work, then none of the interceptors are going to work. So your probability drops down closer to zero. Yeah. Right. I mean, that really is a case where you can add the probabilities. Zero plus zero plus zero equals zero. Yeah, you were about to tell that story about James. I mean, James found it really nice.

I find it very hard to explain to people the notion of independence statistically. But James had a really nice illustration that he gave to Glenn Kessler in The Post.

Aaron 15:51

Yeah, just to quote him out of Kessler's piece in the Washington Post, you know, Acton says, it assumes that the failure modes of the interceptors are independent of one another, but in practice if one interceptor fails because of a design flaw, say, it's much more likely that others will do so for the same reason.

Jeffrey 16:08

And the example he used was if you have a family that gets sick and you start treating them, if the treatment doesn't work for the first family member, It actually does tell you something about the efficacy of the treatment for the others. It's not determinative, but it does reduce the likelihood of effectiveness because family members are related.

Aaron 16:27

Right. Exactly.

Jeffrey 16:29

Yeah. And so things like allergies can run in families, right? So if you have a common problem then you're not necessarily going to get there. Right. And then there's this other piece that I complain about, which is 60%, I think comes from the test record, which is actually about 50 to 55%. But you know what? They don't count.

I know what they don't count, but tell the listeners If they cancel a test due to bad weather.

Aaron 17:01

Right.

Jeffrey 17:02

Or if something goes wrong and they can't do the test that day.

Aaron 17:06

Right. And it's a wash. It doesn't go on its test record. As of the North Koreans, look at the Doppler radar and check out cloud cover before they would fire a first strike.

Jeffrey 17:16

Yeah, it's not like there's somebody with a whistle that blows the whistle and says, no, no, no, no, you can't launch now. That doesn't count. It's raining.

Aaron 17:24

This is not a baseball game. And maybe they would study our own vulnerabilities in time of first strike to those vulnerabilities.

Jeffrey 17:32

Yeah. I mean, we don't even have to talk about scripting. I just think when you look at, if you counted the test record fairly... Included instances where the system was not available, and indeed the evaluators did criticize the system for having low reliability and availability, then I think you're way under 50%.

Aaron 18:00

So why does this matter, right? I mean, because Trump said this Determinative statement as reassurance to the American public against his rhetoric, against the DPRK, that we could control escalation and at least defend from potential escalation getting out of control.

Jeffrey 18:22

Yeah, I think it matters because Well, for two reasons. I mean, one is I definitely worry that the president is going to act on the basis of this belief, right? I mean, we've discussed the best way to understand him is as a low information voter. He is a person who makes decisions based on the poor information that's available to him.

And before people get all angry and complain that we're being partisan, there are plenty of stories about how that's one of Kelly's first tasks as the new chief of staff was to try to keep the garbage information out of the president's head.

Aaron 19:00

Right.

Jeffrey 19:02

And so he might be willing to take very dangerous risks because he doesn't understand that they're risks, right? Because he thinks we can shoot down all of the missiles if he thinks that they're, you know, whatever, they're less than 10 and he has 100% chance of shooting each one down because he's got more than 20 interceptors. And that would be a really dangerous thing to do, even if it works out, you know?

I mean, that's the thing about statistics. Even if he gets lucky or his luck holds, You know, I mean, you're still taking this incredible risk with enormous numbers of human beings.

Aaron 19:40

Yeah. And just to back up about the data, there has been some good studies about this. I mean, the National Academy used to do a very, very good study about this topic. You know, the cast out on basically the system as a whole.

Jeffrey 19:56

Oh, that study, which I find really interesting. It's been totally ignored as far as I can tell. But, you know, the National Academies, I don't know how long ago it was now. It was a couple years ago. I mean, it's recent, but not that recent. I think it had just come out when I'd moved to California. So that was like, maybe it came out in 2011.

I mean, that thing is scathing. It refers to the Missile Defense Agency as having a hobby shop approach.

Aaron 20:26

They're not nice.

Jeffrey 20:27

No. And I, you know, not to talk too far out of school, but I know for a fact that one of the internal discussions they have was to suggest shutting down the Missile Defense Agency entirely and returning missile defenses to the services, which they stopped short of doing. But I think if you read that carefully, you can see between the lines that a significant number of the commissioners just wanted to close the Missile Defense Agency entirely and let the services do missile defense because the program management was so poor.

Aaron 21:00

Yeah, and they talked a lot about this issue. I mean, just go into it.

Jeffrey 21:05

Oh, yeah. So, I mean, they were very scathing about the ground-based mid-course defense system. I mean, you know, the backstory is they got, they were initially impaneled to look at boost phase. Right. Just hitting the missile on the way up. Right. And that's stupid. And nobody thinks that will work except for crazy people. And they got together.

And by the way, these are like missile defense advocates, right? These are not people who hate missile defenses, but they got together. And they spent like, I don't know, some tiny fraction of the time they had allotted. They're like, okay, we all think this is stupid, right? Like, yes. Okay, do we want to double check the math?

Yeah, okay, it's really dumb. Right? Because you can't, you've got to be really close to the missile in order to hit it in the boost phase. And like, the North Koreans are just not going to let you get that close.

Aaron 21:52

Yeah, we've talked about this on other pods.

Jeffrey 21:53

Yeah. And so they expanded their mandate, right? Because they said, look, boost phase isn't going to work. And here's the ugly reality is the ground-based mid-course system in Alaska isn't going to work.

Aaron 22:05

They were scathing on that program.

Jeffrey 22:07

Yeah. And so they suggested... Totally rebuilding the interceptors, adding a third site, and totally rebuilding the radars. And, you know, Congress is just wonderful. I mean, Congress gets this report which basically says the system in Alaska is a disaster. You have to start over. You have to completely replace the interceptors and the radars and probably also where you base them.

And they're like, oh, we need a third site for this.

Aaron 22:33

I was about to say the only thing that they care about is the third site. Is that they want more.

Jeffrey 22:39

And then Kelly Ayotte and, oh gosh, who else? Like basically everybody who thought they might host the East Coast site. Yeah. Well, I wanted – you know. We know.

Aaron 22:51

We know.

Jeffrey 22:53

We know where you wanted it. I really wanted to hear Joe Lieberman saying that they should put those interceptors in Myannis, Connecticut. Yeah.

Aaron 23:06

Of course you would because you want to put billions of dollars for a joke. True what it is a joke that's what the National Academy is saying I mean yeah it doesn't it doesn't so you know like it's not 60% right it's it's it's probably 40% now you're guessing the truth is we don't know we have no idea what it is and it's not independent right it's not independent we're inferring math based upon this and then what we know is that they salve a fire for and then you can sort of figure out how they get it But this is all inferences because we don't really know how it works under realistic testing.

We could, I mean, we don't want to talk about scripting and then why they cancel sometimes. And then that's just a scratch. And so we just don't know.

Jeffrey 23:56

Yeah. I mean, it, it, it to me is, is, it's just, it's just baffling. You know, it's one of these really interesting questions, though. Like, people truly believe that we have the system and that it works.

Aaron 24:17

Well, yeah, it's like a third rail of defense politics is that you have to have strong missile defenses.

Jeffrey 24:22

Yeah, but we don't. I mean, I guess that's the second reason Trump's claim matters, which is that, you know, as somebody who would like to actually have a thin defense against a North Korean launch, you know, I'm not wildly opposed to it. I think there would be some value in having that. It would be nice if it worked.

And that's the part that I can't get my head around, is that there's an enthusiasm for these systems that is independent of their performance. Like, I remember we had this big fight about the site that the Bush administration was going to put in Poland. And they wanted to put a GMD site in Poland, which was not useful.

And... You know, the rest of us were like, but you could, you know, like you could use Aegis, right, to defend Europe. I mean, that would be a much better system because the site in Poland really wouldn't defend Europe at all. It would just defend the United States and not all that well. And, you know, there was just this, like, hatred of systems that work and this desire for systems that didn't.

And I don't fully understand it.

Aaron 25:35

No, neither do I. I mean, ultimately, we did get Aegis in Poland, right?

Jeffrey 25:39

Sort of. Yeah. Well, we do. Right. But they canceled the block to be. But yeah. Although that was actually what the National Academies was criticizing because they think the Iranians could shoot over it.

Aaron 25:51

Mm hmm.

Jeffrey 25:52

You know, so I'm actually thinking about these two different conversations I had in Washington in in the kind of late 1990s, early 2000s, which I think epitomized this difference. And both were with proponents of missile defenses. So I was in one meeting that we had at CSIS and Baker Spring was there and I forget who was giving the talk, but this person was talking about space-based missile defenses.

And sort of looked at Baker and said, hey, Baker, does your report have money for space-based missile defenses? And Baker was like, we have money for everything. Right? And, I mean, it was clear that, you know, his view was sort of we should fund it all and more. Right. But at CSIS, we had written a very pro-missile defense report by someone who later worked for the Bush administration who loved missile defenses.

It was called, like, a blueprint for missile defense. It was by Dan Garay. And I remember asking Dan why space-based defenses, you know, didn't get a more prominent role. And it was great. He was like, well, I want it to work. You know? And I think, you know, like, whatever. Like, I... I think, you know, that was a case where, you know, some people truly want these systems, I think, for the capability they offer.

But I think most of our enthusiasm is just Purely partisan, right, or purely ideological and pretty resistant to evidence that a system doesn't work, which is what I thought made the National Academy's report so interesting, right? Because it said, we love missile defense, but we want it to work. Right. And so we need different interceptors, different radars, and a different architecture.

And everyone was like, does that different architecture mean we could have more of the same crappy interceptors and crappy radars?

Aaron 27:43

Well, I think here at the Arms Control Wonk podcast, we're just pleading for math. Yeah.

Jeffrey 27:48

0.5 to the second power is 100. I don't know what you're talking about. Hey, you know what's easy math, Aaron? It's not 0.5 to the second power. How about 0.5 of a day? Because if you have half a day on Monday, November 6th, the Cato Institute invites you to participate in a conference. What are we going to do about North Korea?

It's going to be held at their building in Washington, D.C. With the North Korean crisis now one of America's top foreign policy priorities and no immediate answer in sight, this free conference, including a keynote address by former governor and North Korean negotiator Bill Richardson, brings together experts who will be discussing what approaches should be taken, the state of North Korea's nuclear capabilities, how to maneuver through the escalating war of rhetoric and provocative actions, and more.

Attend in person or watch the conference live online. For all the details, go to kato.org forward slash North Korea.

Scott 28:50

And they can follow that same link to RSVP. You'll need RSVP to actually attend the conference. So head over to kato.org forward slash North Korea to RSVP.

Jeffrey 28:59

Yeah. Hey, Scott, how are things doing on the Slack channel?

Scott 29:02

The Slack channel is excellent. If you would like to be part of the Arms Control Wonk Slack channel and exchange views, opinions, documents, satellite imagery, anything, head on over to patreon.com forward slash acwpodcast. At $5 per month, you can get access to our Slack channel, and that's $3 per month if you have a .edu or other student-related email address.

And if you support at the higher levels, like $15, we'll send you some nice physical merchandise from the Arms Control Want collection. Hey, Aaron.

Aaron 29:35

Yes?

Scott 29:36

Thanks for joining me.

Aaron 29:38

Always a pleasure. And thank you for listening.