Visualizations and Verification

Jeffrey Lewis 0:06

You are listening to the Arms Control Wonk Podcast, a podcast on arms control disarmament and nonproliferation. I'm Jeffrey Lewis, the founding publisher of the Arms Control Wonk blog and a professor at the Middlebury Institute of International Studies at Monterey. I'm Aaron Stein, director of the Middle East Program at the Foreign Policy Research Institute.

Anne Pellegrino 0:26

And I'm Anne Pellegrino, a research associate at the Middlebury Institute of International Studies at Monterey. Aaron and Anne, you know what I say?

Jeffrey 0:35

No?

Aaron Stein 0:36

You say a lot.

Jeffrey 0:37

Well, I say a lot. Well, one thing I say is the more the merrier. And we have a guest. We have James Palmer from the Atomic Weapons Establishment in the United Kingdom. Hi, James.

James Palmer 0:51

Hello, Jeffrey. Hello, Anne. Hello, Scott. Thank you very much for having me.

Jeffrey 0:56

We've invited James on to talk to us, not about the BOMI bits that AWE does, although they're quite BOMI, but about data visualization, because we just had this big conference here in Monterey where we brought All of these various tech companies down to talk about the ways in which we could take their software and make cool nonproliferation use cases.

And in some strange way, almost all of this was inspired by meeting James in Vienna at the CTBTO's S&T conference, where he showed us a cool program called GeoVisionary. Isn't that right?

James 1:36

That's correct.

Jeffrey 1:37

That's correct. So welcome, James. It's good to have you. And your plummy British accent.

Aaron 1:46

Class the pot up immediately just by saying a few words. It makes my tea taste better.

Anne 1:54

We have quite the role reversal this morning. Jeffrey's drinking tea and James is drinking coffee.

James 2:00

Well, it's good not to live up to stereotypes sometimes.

Anne 2:06

James, it was so great to see you again in Monterey for the conference. And what you showed off I think was really awesome. And it was something that actually Jeffrey and I saw in Vienna. So for our listeners who either don't remember when we gushed about your presentation way back when, can you give a little synopsis about what exactly you have done with 3D visualization?

James 2:35

Yes, of course. So the presentation I gave at the S&T was some work that was inspired after the event in the North Korean test site where they invited journalists in and we for the first time had Photography and video from the ground and the North Koreans showed a poster to the journalists which was obviously captured in the imagery that contained a map which showed where they said the tests occurred and where things like the tunnel entrances or the adits for the test site where they were located.

Jeffrey 3:13

This, by the way, made us super jealous because we had made exactly the same map, but ours did not render nearly as nicely as yours.

James 3:25

Thank you. So basically, I was already working with the GeoVisionary software, which you can imagine is a bit like a sandbox for three-dimensional data, in particular geospatial data or 3D CAD models. So I can throw data in and it renders it in 3D and I can fly around it in 3D and I can display it with a projector with depth perception, with flashing glass, the old glasses that you get a bit like in the cinema.

All the way through to putting a virtual reality immersive system on and wandering around the location. So during the Monterey workshop, Anne got the opportunity to have a quick hike up Mount Mantrap whilst wearing a virtual reality system, which is pretty cool. But the reason for creating all of this was the question, well, James, if you can geolocate the map and give the coordinates for the indicated tests to our seismometry group, Then they could calculate the relative positions of the tests from the seismic signals and we could compare the two.

And so the presentation was the summary of that work and some side work that came out of that.

Anne 4:48

So when I was able to trudge up Mount Mantap in the VR headset, I have to say that that was one of the trippiest experiences ever. And then also when I was in the flame bucket of the large engine test stand at SOHE, I actually – Virtually fell off and into the flame bucket, and I will say that it actually felt like I was falling. I was able to climb out quite easily, but I, you know, we have tried really hard to kind of enhance our virtual reality capabilities, and so we created, you know, the the Chamjin Missile Factory Museum, which is always a hit, but The visualizations that I stepped into in terms of what you created, I think, were incredibly powerful.

And I actually felt like I was there. And I think that that is... It showed me what is possible in terms of what we at CNS hope to do with our 3D visualizations because I felt like I could actually be there and that then allows me to do better analysis of a site. I can't actually go to these sites in North Korea or sites that we want to look at in Iran.

I physically can't be there. And so how do I enhance my analytical capabilities from my office? And I think that what you have done really displays what's possible.

James 6:24

Well, thank you very much for that. And thank you for basically saying everything I might have been able to say in relation to why this is a good idea and why... Well, it's a useful tool. But that's no bad thing. Because I think part of the power of the visualization techniques that are available to people other than just film studios and governments these days, allows that sort of message to be conveyed through experience rather than someone telling you, this is what you should think about the thing that you're about to experience.

Jeffrey 6:58

For listeners, I mean, one of the issues that we have been running into is that we have these enormous data sets. So we have terrain models. We have 3D models of facilities that we've built. We have really high-resolution satellite images. We have ground truth images. And it seems like you have all this information, but you really need a sandbox where you can bring all your data to play.

And it's really not a trivial thing for that sandbox to be able to serve up and present all that data in an integrated way. So when we did our first model of the North Korean nuclear test site, we had wanted a high-resolution satellite photograph, an accurate terrain map, accurate tunnels, and we wanted to be able to walk through the tunnels.

And I remember being told by a research assistant, like, uh-uh, you can't have all those. If you want the accurate terrain map and you want the – The satellite image that's going to take up so much computing power that I am just never going to be able to build a two scale tunnel. So, you know, like imagine me sitting in Vienna and actually seeing this thing which is the thing that I was told I couldn't have.

Yeah. I don't like to be told I can't have things.

Aaron 8:19

I actually want to know how giddy you were because I can imagine you just getting like really excited and sort of squirming around in your seat way in the corner of James after his presentation. Is that how it went or did you meet before the presentation?

Anne 8:29

It was exactly that. So James and I actually presented – well, it was the same grouping. So I – James was last up and I was – I went right before him and so I presented on my Novia Zemlya research page. And then I had already seen, obviously, the schedule. So I knew to watch his presentation closely.

But then when we were watching the presentation, it was like, oh, like we tried to do this. We went about it in a completely different way. And here is this tool that we are now salivating over because this makes everything that was not possible for us possible. So we cornered him.

Jeffrey 9:09

Immediately after the presentations, and what we talked for like an hour and a half at the conference, and then we took him out for beers. Yeah, although it was, it was giddy, it was glee mixed with fury, because, because having been told it wasn't possible, and then to see him stand up there and say like, oh no, it's totally possible, it was like ah

James 9:34

Yes, I got an inkling of your enthusiasm for it. Fortunately, I had the laptop with me and we were able to look at the model of the test site live afterwards and fly around it live with the laptop. And again, there are other tools out there. The fact that I use GeoVisionary isn't, you know, I guess at official levels.

Jeffrey 10:00

But I think GeoVisionary and Cesium are both really excellent sandboxes.

James 10:05

Yes, and there's a few more out there because I investigated a few before I settled on GeoVisionary. And I had a very similar experience to you when I first saw GeoVisionary being demonstrated because the salesperson was running it pretty much just as you saw me running it during the coffee breaks in Vienna of 2019. Running from a laptop onto a screen and just showing the program doing its thing.

At that point, I saw it and I thought that could do the job that I need a tool to do, because at the time I was doing work in support of the Comprehensive Test Ban Treaty Organization in relation to their on-site inspections. And for all of the same reasons, bringing lots of data together, looking at it where the data was collected, it seemed to be able to do the thing.

Aaron 10:56

Yeah, let me just jump in right there because I know that at least we were talking yesterday. I had the chance to talk to you for a few minutes yesterday as we were preparing for this podcast. And you brought a lot of your verification skills using these types of models to a CTBTO exercise in Jordan, if I'm remembering correctly.

I was wondering if you could just go into detail a little bit about that, how you've All the verification stuff associated with it.

James 11:17

Okay, yeah. So for those that don't know about it, the Comprehensive Test Ban Treaty Organization has, if the treaty comes into force, has a remit to do on-site inspections if certain things occur. No time to go into the detail of those, but in 2014 they carried out a very large exercise to test as many of their capabilities to do the things needed for an on-site inspection as possible, which involved several hundred people, the movement of over 100 tons of equipment to a location in Jordan, and they had to search for evidence that did or did not indicate that the fictional country of Meridia had or had not infringed the treaty.

Jeffrey 12:03

I wish I could name these fictional countries, by the way. Meridia's fine, but I think I'd go edgier.

James 12:12

We know. So then... Again, similar to the North Korean test site and others, there's lots of data collected, seismometry, magnetometry, ground resistivity, tomography. That's collected that the inspectors need to obviously collect, analyze, but also understand and interpret. And I was there for a week as one of the UK's official observers and I got to see this happening live.

And I could see that the data was there, but Because you often had to look at the data in printed out pieces of paper or in PowerPoints, different techniques or different data sets that were collected from the same physical location couldn't be seen in the location where they were collected. And I thought it would be more useful to be able to see all of those data sets at the same place because then you can draw out the connections.

Jeffrey 13:19

Yeah, you know, the thing that really struck me about the reaction you got, and I guess I should let you tell the story, but I think even for someone who participates in the inspection, when you're there and you're kind of immersed in it, but you're kind of locked to the ground by vehicles or your ability to walk, You know, you don't necessarily see things in the same comprehensive way that you would if you had a model, particularly if you could walk through that model before you went to the place.

James 13:52

Absolutely. So and I think this is one of the great utilities for being able to do this sort of 3d visualization, if it can be done live in an on site inspection. And from a hardware and technical perspective, there's no reason it couldn't. Because I myself as part of the as my role as an observer, I went to the site and walked around the disused quarry That was actually where, according to the exercise, the surface ground zero for the infringing detonation occurred.

And as I walked round on the top of the quarry and then at the bottom of the quarry, I thought I had a good understanding of the the quarry layout and it wasn't until I saw a single aerial photograph of the quarry that I suddenly realized that I'd missed about a third of the floor area of the quarry simply because of the view angles when I was up high and down low.

The two basically formed a blind spot of this whole plateau in the quarry. Which obviously became clear when I saw an aerial photograph. But then afterwards, when I was able to stitch all of the aerial photography together and recreate the terrain and then wrap high resolution imagery onto it and then put all of that into a geo visionary and fly around it for real.

Well, for real in the digital world. As you say, it would have been a fantastic tool for inspectors pre-mission or even the inspectors who never actually got to leave the base camp because of their roles to be able to literally see what this random location in this inspection area looked like both in structure and in appearance.

Jeffrey 15:37

This is the thing we run into all the time with visualizations, which is people see them and they associate them with being pretty, which they are, but they don't often see that there is a kind of value that is intellectual. So we, we have this with 3d images all the time. Like people will think they're a parlor trick, which they kind of are because they are cool, but you see the photograph differently after you've seen it in 3d.

And I think whether you're doing that with a stereoscope or whether you're doing that with a computer generated model My ability to look at a satellite picture of a site is completely different once my brain is adding the terrain and once my brain knows what the structures look like. And so I just think these kinds of visualizations, whether you're using GeoVisionary or whether you're using Cesium, are incredibly important to doing the work we all do better.

Anne 16:33

I completely agree. And so I guess that still is a pretty good kind of segue into my curiosity about what the reaction was, James, that you experienced to your visualizations, because it strikes me as something that is because it is new, it can be either embraced or it can be kind of written off as, oh, we don't actually need this.

So curious about the traction that you had.

James 17:03

So in general, when people experience it, the most negative response I've had from demoing the systems has pretty much only been when using the virtual reality system. And that's simply because people say, I don't like using those things because they make me feel queasy, which is perfectly reasonable because there's lots of reasons why people feel queasy when using those.

And some of that has got better now with the better computer technology. So the imagery is much smoother, but things are more complicated than that. The reasons behind why people get motion sick in general. But certainly, generally it's very positive. Part of the challenge is working out how to integrate A capability no matter where it comes from into, say, pre-existing workflows for how people do their work.

There is also some discussion around they might already have capability in one area, producing things with, because there are various ways you can build these interactive 3D worlds. The program that I use is an off-the-shelf package, and you do everything from within the package. Lots of people produce things using games engine technology where they produce separate programs that can then be distributed.

And they're different tools for different jobs.

Jeffrey 18:38

You know, what I really loved in your presentation was that you 3D printed the quarry.

Anne 18:44

Yeah.

James 18:46

Yes. So from my perspective, one of the great things about working in visualization today, you have access to an amazing variety of data sets that you can bring together. And if you have the right tools, you can bring them together in the same place. And from my perspective, the two gold standards for being able to present information is either virtual reality, but that has limitations generally in that it's An individual person has that experience.

The other gold standard, from my perspective, is a projected image that has active 3D stereo. So you put the glasses on and then your projector screen turns into a window. And that can be amazing. So I've seen that with cameras. The British Geological Survey, who were partly responsible for the creation of GeoVisionary, who worked with a UK company called Vitalis to produce it, they have a whole wall that's back-projected at 4K resolution.

That they can turn on this active stereo and the whole wall, when it works and they turn it on, the wall just drops away and you're looking through a window. And that's for showing multiple people. That's fantastic.

Aaron 20:07

With the ability to bring things to life, as you were just talking about, with the 3D rendering and the ability to project them and make walls fall away, at least visually, so that you can... Physically, or at least experience things that are modeled. I'm curious, you applied this to verification with the exercise with the CTPTO, but there could be so many other applications, both for people like Jeffrey and Anne who do this stuff more full-time in the open source world.

But also for governments as we seem to – as they work through or grapple with verifications of potential future arms control treaties or something along those lines. I'm wondering what you think about that, how we could think about using this stuff for – this is an arms control podcast for arms control or for making things more transparent.

James 20:56

Okay, so – What we've spoken about at the moment has been sort of looking at things that are remote sites and accessible sites or remote site and accessible in terms of the CTBTO inspection. When I was at CNS, they were kind enough to host me for a seminar and one of the case studies I presented at the seminar was An activity that we had here in the UK called Letterpress, which brought everything to a much more local level with hands-on, very much at the coalface.

So it was a nuclear weapons disarmament verification simulation, and it was carried out by what's called the Quadrilateral Nuclear Verification Partnership, which fortunately has been shortened to the Quad, which is formed of the UK, the USA, Norway and Sweden. So that's quite a novelty because you've got two nuclear weapon states and two non-nuclear weapon states working on this verification activity.

And the objective was to carry out nuclear weapons verification simulation in a representative nuclear weapons facility. Using non-proliferative but representative treaty items. So, apparently the first instance of its kind, and it was designed to enable non-nuclear and nuclear weapons states to work together in an arms control scenario so that we could look at developing techniques, procedures, and methods that could be used to then inform future nuclear weapons treaty verification regimes that could involve non-nuclear weapon states.

And so the simulation itself formed part of a wider scenario in which two nuclear weapon states have agreed to a significant reduction in their respective nuclear weapons stockpiles and have invited two neighboring non-nuclear weapon states to be part of the inspectorate task for the technical verification of the state declarations. So the really cool thing about it was that the simulation took place at RAF Honington back in 2017, where it used Genuine former nuclear weapon storage bunkers in the simulation and ballistic casings of retired nuclear weapon systems to enhance the realism of the exercise.

So As much realism could possibly be brought into the simulation, and it was a simulation, not an exercise, because you exercise capability, but this was all about the work that you need to do to even create the capability. So activities... Down to you know, as part of that, tags and seals were applied, various techniques were used, and sealing mechanisms, accounting, tracking things. There was the movement of a weapon from a storage facility to a dismantlement facility, which was then to be verified as part of this wider treaty, and it was looking at Exactly.

How do you go about doing that?

Jeffrey 24:00

This, by the way, is so much more responsible than what we do with this stuff because we just post selfies of ourselves with the bombs that we create virtually.

Anne 24:09

Yeah. I always struggle with how kind of catchy and fun it is on Twitter. And then I have to think about like... Okay, so I'm pretending to tinker with North Korea's thermonuclear device. Is that how I should best be spending my time with this software? And the answer is yes. Oh, what?

Jeffrey 24:39

I expect to see a missile on your Christmas card, Anne.

Anne 24:42

Oh my gosh, I can make that happen.

Jeffrey 24:45

I'm sorry, James, I interrupted you.

James 24:46

That's okay. So getting back on topic. So this activity happened, and obviously we had people coming from all over the world. It had to be planned, the execution of it, people who were playing the part. So in that, I played a member of the host team. And again, to make it as... Productive an activity as possible both the non-nuclear weapon and nuclear weapon country representatives were on the host team as well as on the inspectorate team and there was an amazing amount of learning that occurred during that activity and afterwards again I thought well It takes time for people to come, literally to come to the same location, to just look at the location.

And it would have been useful for, certainly at the most basic level, for the people playing the parts of the hosts, To have been familiar with this location before we went there to play the parts as hosts. And a number of us had never been there before. So, of course, as a host, you're like, well, you know, take the inspectors from these offices to bunker 12.

And you then have to flick through your piece of paper and find the map and try and work out, oh, how exactly do I walk from here to bunker 12? Where if it were real, then of course you would know exactly where bunker 12 was. And I thought, well, if I recreate as much as possible the physical location and some of the objects that were used in the letterpress simulation, What cool things can we do?

At the very least, we can do site familiarization for future activities. It could be used for planning. And one of the things it's also turned out to be amazingly useful for is briefings. Because as Anne and Jeffrey have both been lucky enough to see, because I had that model running when I was at the workshop, You can let people explore in either on the screen or in virtual reality the location for five minutes, and they know more about the layout than if they had spent two hours flicking through maps, photographs, videos, and PowerPoints.

Jeffrey 27:14

That's what I was getting at when I asked you about the level of familiarity you felt with Jordan, because I thought you at one point had said that someone who was involved, someone else who was involved with the inspection... Had that same reaction, that the model was significantly better than the preparation and having been there, that it just really helped organize intellectually all of the different bits of data.

James 27:40

Am I misremembering that? No, no, you're correct. But it has been true on both models that I've built. So that particular statement came out of me generating the material, the stills and in particular the animations that I used for a talk I gave with a co-author at the CTBTO Science and Technology Conference in 2017. Where I used the model that I built for the IFE14 area to give two fake briefings to demonstrate this is what you could do with the tool, and I'm going to show you by doing, not by telling.

And so the first one was to show, well, what could you tell people, the inspectors, before they went into country using only information that was available before they started the inspection? And it started off looking at the data that was available, which was low resolution terrain data, but with some high resolution satellite imagery from Like low Earth orbit and above, and the classic view that you'd see with the printed out map, but then flying down and over the surface.

And in that 10 seconds, you learn more about the appearance and the topography of that area. Then pouring over a map with contours for hours. And in that case, you immediately saw that, you know what, it's a very rugged landscape. So the logistics people would be instantly thinking, it's going to take a long time simply to drive around this area and start factoring that into their planning.

And that's with 20 seconds worth of footage.

Anne 29:31

You know what I really want to make a model of for verification purposes? No.

Jeffrey 29:37

I would love to throw all of our data about Novaya Zemlya in. Into a program like it was a serious answer. I was, I was thinking you were gonna model like, like your neighborhood and all the places where your dog pooped with little, you know

Aaron 30:02

I was thinking something about drinking, like modeling my local bar so I know which way to turn after one too many.

Anne 30:10

Y'all, I'm disappointed in you. If there's anything that you know about me, it's that I have this affinity for this site up on Novaya Zemlya.

Jeffrey 30:19

It's true. It would be amazing. And, by the way, much better than the bet we have in which you'd have to be paid a million dollars to live there for a year.

Anne 30:29

Under the right circumstances, yes. And a very nice mansion with a panoramic view across the street. Nope. Maybe. Nope. Nope. Nope. No, okay. But in all seriousness, that is a site that strikes me as a place where –

Jeffrey 30:54

It's true because it's massive, right?

Anne 30:56

Right. It is massive. There are things that have gone up there in recent – in the last decade that have caused the United States a lot of heartburn. And so I think that allowing people to walk down the massive valley and to see kind of the scale of the place would be one aspect. I would just like totally geek out.

But I think it would be quite valuable.

Jeffrey 31:26

Could we do like a virtual small gauge rail thing? You know, like Disneyland.

Anne 31:34

Well, so the great thing about the VR headset is that you can literally just click and fly to a point.

Jeffrey 31:41

I know, but I just think it would be great to have to get in a little Disney train.

Anne 31:46

Oh, yeah. I was thinking I could ski. And here on the right, we have the landslide from the explosion that occurred in Tunnel B1.

Jeffrey 31:59

We could just have like a dead Soviet nuclear weaponier's hand sticking out of the rubble.

Anne 32:05

I would gladly lend my voice to a – nothing surprises me from you anymore, Jeffrey. So I just kind of like – it didn't even register. It didn't even register to me.

Jeffrey 32:17

I think we are coming to the end on time because we're at 39 minutes.

James 32:24

Okay, one quick observation then is to cater to your inner child, Jeffrey, and Disney Railways. It would be effectively, I have kind of done that within the software because you can load in GPS tracks with the software and you can then associate a model to that track and click it and the 3D model, whatever it is, will then follow the GPS track.

In real time or accelerated time. And you can also lock the camera in the software to that GPS track. So yes, you could have your little, you could literally see the view from your Disney small track train as it trundles around a three-dimensional GPS track that you can create to go wherever you wish.

Aaron 33:19

Sold. To say, and you got another project you can do here. Well, James, you know, that was fun. I, I didn't get to see the thing in person and I don't have 3D glasses at my house. For, for listeners, you actually did share your presentation, so we were walking around the North Korean test site during this and we all were oohing and aahing, even though I think you guys, you and Jeffrey had seen it before So thanks for coming on and Jeffrey thanks for obviously for, for joining us or at least leading the conversation.

And thanks everybody for listening. If you enjoyed the show, head on over to patrion.com slash ACW podcast. Whereas for as little as $3 a month, if you were a student or $5 a month, if you are not, you can join our Slack channel and There's more very cool open source stuff in there. So with that, thank you, everybody, for listening.

And thank you again, James.

James 34:16

Well, thank you very much for having me. It's been a pleasure.