Planetary Radio: Space Exploration, Astronomy and Science - Space Policy Edition #16: NASA's Flagship Missions: Are They Worth It?

Episode Date: August 31, 2017

The multi-billion dollar, multi-decade Cassini mission is about to end. A new report tries to answer an important question: are flagship science missions like Cassini worth the effort and expense?See omnystudio.com/listener for privacy information.See omnystudio.com/listener for privacy information.

This is the Space Policy Edition of Planetary Radio, back with number 16, SPE-16, for September of 2017, with much more to talk about and the people that we'll be talking with, my regular cohorts on this, the Director of Space Policy for the Planetary Society, Casey Dreyer, and the Space Policy Advisor living there within the Beltway in Washington, D.C., Jason Callahan. Welcome, gentlemen. Hey, Matt. Hey, guys. Good to hear from you again.

I hope that you're doing well, and I'm dying to hear where you were for the eclipse. Casey? I went to western Oregon, just west of Salem on a extended family person's farm who was very generous to allow 60 people to descend and watch the eclipse. Perfect skies, spectacular experience, and just camping for the weekend and enjoying celestial alignments. Unusual for me. You saw the corona? Oh yeah, absolutely. It was huge. And Venus. And I saw Mars. All the good stuff. And yeah, it was great. I didn't even get a sunburn.

Yeah, I was staring at the sun for three hours. Well, I'll get to my somewhat sadder story in a moment. But Jason, where were you? My wife and I went down to Columbia, South Carolina to visit her parents, who had driven up from Savannah. And we stayed at a hotel there with a pool so that our daughter could spend lots of time swimming and paying less attention to the eclipse than we were. But we brought our solar telescope, had very intermittent cloud cover early in the event, but by the time we got to totality,

it was absolutely perfect and beautiful and wonderful. We had a really good time. Okay, well, I'm intensely envious because, as you may have heard, I was in Carbondale at Southern Illinois University, Carbondale, which put on the show of a lifetime for the Great American Eclipse. But during the five-hour show, 100 degrees, 90-plus humidity weather, we had hazy but clear skies up until about 20 minutes before totality, and then this evil cloud moves in. And it broke up just enough that we had maybe 30 seconds of totality, saw the diamond ring, never saw the corona, 14,000 screaming people who took it pretty well.

And it was fun to lead them. It was an absolute blast. It was a fantastic several days, because we did Planetary Radio Live there as well on Eclipse Eve. But yeah, totality could have been better from SIU Carbondale. Well, Matt, you know what this means. You'll have to go down to Antarctica to catch the solar eclipse coming up in the next couple of years. I was thinking of going back to the Atacama in Chile

where there are no clouds ever. Yeah, go to the place where things don't live. Yes, right, exactly. And humans have to carry a can of oxygen, which I actually did down there. Well, I congratulate you guys on your great experience. Let's talk about what else is happening in space policy. And this is going to be a quick

review up front, just of where we stand in terms of the 2018 budget for NASA, and maybe any news, any updates that you've heard about who might be running the agency before too long. And I should say that we're recording this a few days earlier than we normally do. So who knows what may happen by the time we post this show? Well, fortunately, this will be a quick discussion because the big budget news is that there is no budget news really since the last time we spoke. We haven't seen any sort of compromise budget come out from the House or the Senate for NASA's 2018 budget cycle. We are barreling towards the end of this fiscal year. I think Congress comes back in early September. They will have 12 legislative days to decide

how to fund the government and also to raise the debt ceiling and to extend the Children's Health Insurance Program and a variety of other fiscal cliffs, let's call them. So I believe NASA funding will be relatively low on that priority scale. But no news there. We have heard rumors about a NASA administrator, Jim Bridenstine. Those are as yet unconfirmed because we haven't had a formal nomination. We also have a deputy name floating around, but we'll talk about them when we have an actual formal commitment from this administration, which, as you said, could come tomorrow, could come next year. Who knows? So we're following that rather closely. And that's just our quick review right there.

Jim Bridenstine, remind us, by the way, Republican congressman from where? Oklahoma. Okay. And somebody who has followed space pretty closely, right? Yeah. Okay. And somebody who has followed space pretty closely, right? Yeah. He is actually, he's on the House Science Committee, on the Space Subcommittee, and he has released a variety of actual interesting space policy legislation called the American Space Renaissance Act. It was meant to be more of a statement than an actual piece of legislation that was going to pass. You can look that up online. He's very interested in incorporating kind of a commercial data into weather forecasting and Earth observation and a variety of other things. Hasn't touched as much on the science stuff, but that's why we

talked to his office to always encourage the great science part of NASA exploration, which actually is kind of what we're going to be talking about today. And on to that main course for today's Space Policy Edition. Just a little preview, we're going to start with a bit of a discussion about Cassini as we rapidly approach the grand finale of that mission. More about that in a moment. And then on to our truly main course topic, and that's talking about this report that has just been issued about the place of really big missions, so-called flagship missions with NASA, and a special conversation that we're going to bring you that Jason Callahan had with the co-chair of the group that developed that report. Before we do that, a reminder. We hope you're considering, we've been asking you about this every time we've done the Space Policy Edition,

maybe it's really time now, if you haven't done it already, to become a member of the Planetary Society and stand behind the great work that Casey and Jason do and all of our other colleagues at the Planetary Society. Also, no small thing, help enable us to bring you this show on a monthly basis and the weekly Planetary Radio for that matter. Please consider becoming a member. It's inexpensive and has a reach that will go far beyond your normal grasp. You can do it at planetary.org slash membership. Gentlemen?

Always good to emphasize that the Planetary Society literally depends on individual small donors, our members, to live as an organization. We live and die by our membership, and that gives us this independence and flexibility to pursue and represent you in Washington, D.C., to pursue space science and exploration. We don't have major corporate money that really directs where we go. The vast majority of our funding comes from individuals, and the vast majority of that funding comes from small donations through your membership. So planetary.org slash membership really does make a difference. Every single one at four bucks a month is what it starts at. That's cents a day. So it's a really valuable investment that you make. And we really

use that to the maximum extent that we can. And I would say, Jason, you hear this all the time, the Planetary Society really punches above its weight in terms of the actual organizational yearly budget that we have. We way over represent that because we're such an efficient, focused organization. Casey, actually, we see evidence of that all the time here in D.C. Our colleague Matt Renninger and I spend a lot of time on the Hill and talking to people at relevant agencies and in the scientific community. And we've reached a point where actually people reach out to us to help them with issues that they think we can aid them with. I don't see that happening a whole lot with other groups. It's sort of a unique capability that we have, and it's all made possible by the donations

of our members. And that's just the space advocacy side of what we do. Of course, we advocate, but we also create and we educate. And then we have, guys, I forget which one of you realized this, but it's our own flagship mission, and that's LightSail. With all of the coordinate problems and opportunities that every flagship mission has, it's a big mission for a CubeSat and for a small organization, but it's going to be spectacular. It's huge for us. There is a campaign going on right now as we enter the final months before we hope to see LightSail 2 up on top of a Falcon Heavy, either the second or the third of those big new rockets that will be launched by SpaceX. We are looking for help to make sure that we reach that in the kind of shape that we want LightSail to be in and that we're ready here on the ground to monitor that mission.

There is a campaign underway right now, planetary.org slash lightsail. And at the moment, there is a $50,000 match from a very generous member so that any dollars that you send our way will be matched out of that member's largesse. So we encourage you to look into that as well. Planetary.org slash membership is a good place to start. But then take a look at planetary.org slash lightsail and learn more about that project as well. All right, gentlemen, let's get into the content for this week, beginning with that grand finale that is almost upon us. Cassini.

So we just wanted to just acknowledge this a little bit. I'm preparing an event here in the Northwest. We're doing a wake for the Cassini spacecraft. It's going to be a lot of fun. It'll be a little sad, but as most wakes are, we'll celebrate example of a flagship science mission. It is over a $3 billion cost, I think, adjusted for inflation. It's about $3.5 billion that NASA spent on it. It had a major contribution from the European Space Agency and the Huygens probe, which landed on Titan, of course, back in 2005. But I did a little digging for a future article coming out in the next planetary report. And I found that the very you can kind of trace the Cassini mission back to a joint working group. And, you know, every spacecraft, I feel like begins with some sort of joint working group or committee meeting. It was in 1982, right after the survival crisis of planetary science.

I believe we've talked about that before, this is at about the lowest level of funding for planetary exploration the U.S. has ever gone through. NASA and European Space Agency had a joint working group to say, how can we work together to explore the solar system? And one of the top recommendations from that joint working group ultimately was a Saturn orbiter and Titan probe, 1982. From there, it was just a short hop, skip, and a jump 15 years to get a mission on the rocket sitting in Cape Canaveral in 1997. I also will post in the show notes,

I found it took a lot of work to get from that original group to actually making a spacecraft to funding it, of course. And we found what's called a Phase A study, kind of a very in-depth study that was done by NASA, released in 1988, which basically outlined the mission that we saw. And then from 1988, it took, of course, nine more years to actually build the thing. And then from the launch, it was seven years just to get to Saturn in 2004.

And we had 13 brilliant years at Saturn really understanding the dynamics of that insanely complex and beautiful planet. And we have to just appreciate that amount of effort it took. It was a 30-year effort to get to that point. The science returned by this mission has been worth decades and decades and will go on for decades as scientists work with that data. Adjusted to 2016 dollars, the official life cycle cost, according to NASA, at the 2004 launch would be about $4.1 billion. Wow. Oh, OK. That's life cycle with operations. Yeah, I was thinking of just development. Planetary missions actually rarely peak above that level.

I think the only missions that cost more were Viking back in the early 70s when you adjusted for inflation. And I don't remember if Voyager actually peaks above $3 billion ultimately when that's adjusted. We actually have a list in our upcoming paper that we will discuss about this. But my point really is that this was a big mission. It took decades to put together. It gave an incredible amount of science. And don't forget, it's the only mission ever to orbit Saturn. And it was the first mission to arrive at Saturn since 1981

when Voyager 2 flew by it. As Cassini ends, it actually frees up a significant amount of funding internally to NASA's Planetary Science Division that NASA has to pay, obviously, the operating costs of these big flagship missions. And Cassini was about, give or take, $58 million a year for the last few years. And that $58 million gets kind of get rolled back

into, let's say, a future mission that goes to another planet, maybe with an icy moon itself. It helps kind of absorbed into the Europa Clipper project. I call that a bargain. At least we certainly hope that that's where the money goes. Right. Yeah. There's no guarantee. And also just one thing to discuss about that, as we were prepping for this episode, Cassini has been such a success. And we kind of take it for granted that once you have made a spacecraft and it's there and it's taken so long to get there and you spent decades working on it and it's collecting data and working beautifully that NASA will just continue to pay it. But I was reminded of back in 2013 when we were having some really tough budget times for planetary science. There was serious talk internal to NASA to actually prematurely end the Cassini mission three years ago. There was a lot of internal pushback to that.

We pushed back internally to that. And there was a question, though, that the fact that we have Cassini diving through the rings right now in its grand finale was not a foregone conclusion. And that took effort and argument. You know, NASA spent almost $200 million between then and now just to continue that mission going. It is a non-trivial cost. And I'm really glad they did. But people have to fight to continue these big missions, even when they're producing amazing, unique, irreplaceable science. $200 million over four years in a NASA budget, that's the cost of developing a smaller medium-class mission. Yeah, like an Astrofix extension, that's about what TESS is going to cost,

the Transatlantic Survey Telescope. So it becomes a question, as you're making that decision, is it better to have an operating spacecraft that you know will function, or is it better to develop something new that goes someplace more interesting or equally interesting? Generally speaking, NASA tends towards the lower risk capability, which would be the asset that you already have. But that's not always going to be true. So it's a question worth asking. Yeah, the spacecraft in your hand versus the two in the bush question. And this brings up this larger question, obviously,

that we want to talk about more today, which is, are flagships worth it? Before we go into that, I do want to mention, some people may be thinking, when are we going to go back to Saturn? Or how, if we will, we being, let's say, humans. NASA really, I think at this point, is the only space agency that has the technical capability to go that far. We have no clear pathway. I think the only opportunities are that the next time that NASA is going to be competing with their mid-class, their mid-sized planetary exploration mission. It's called New Frontiers. Juno is a New Frontiers mission.

OSIRIS-REx is a New Frontiers mission. New Horizons was a New Frontiers mission. Their cost capped at roughly a billion dollars, including the launch capability. NASA is deciding on the next selection right now. I think 12 proposals have been put in for the New Frontiers selection. NASA will select two or three of those for further study at the end of this year. And ultimately, by mid-2019, NASA is going to select one of those proposals to launch in 2024, 2025. A variety of them are actually missions to the Saturn system.

Sprite would actually take a probe into Saturn's atmosphere, kind of like Galileo had. We have a mission called Oceanus, which would go to Titan and map Titan at very high resolution and try to study its chemistry and geology. And kind of my personal favorite would be Dragonfly, which is kind of a helicopter drone that would land at multiple spots on Titan.

It would land and then kind of hop around to one area to another, sampling the surface and looking for habitability, which is a pretty awesome one. I have no idea if they can make that actually fit. This is what NASA will actually decide what the risk posture of that would be. But there are several missions, but again, no guarantee. And even if I was looking at the proposals for each one of these missions, even if one of those is selected, Saturn's far away. We don't have this.

What did Cassini launch on? A Titan Centaur? I believe a Titan. And we don't have those rockets anymore. And so the fastest you could get on conventional rockets that we have now, not counting the SLS, would be about a 10-year travel time. If everything goes exactly right and NASA chooses another Saturn mission, the earliest we're looking at before we see images or data coming back from the Saturn system

would be 2034, 17 years from now. So eat your vegetables, everybody. That all brings up a really interesting point. How exactly does NASA select missions? And this bleeds directly into our main topic today. There are two processes by which NASA selects planetary science missions. There are competed missions, which is what Casey was just talking about with the Discovery Program for small class missions and the New Frontiers Program for medium class missions. And then there are strategic missions, which are not

inherently large or flagship missions. There are strategic missions that are smaller than those, but generally speaking, all large missions are strategic. Now, the difference between those two is that the competed missions, the categories of missions are determined through the decadal survey process, looking at the most interesting scientific questions for each of those class of missions. But then people propose different missions to any of a number of destinations determined through the decadal survey. And then NASA convenes selection boards of community experts that go through and evaluate each of those proposals to determine which mission is the best.

And then NASA selects that, pays for it, develops it, and launches it. The strategic missions are very different. The questions that the strategic missions address are want to lead the mission. And NASA runs those missions basically soup to nuts, whereas competed missions are run by the principal investigator who oftentimes is outside of NASA. They assign a mission to a NASA center as opposed to a person being responsible for it. Absolutely. What we're talking about today are strategic missions and specifically the large strategic

missions. NASA defines those these days as basically anything over a billion dollars or a billion and a half dollars. We'll get into that more in my interview. But generally speaking at NASA, that's how you define a large flagship class mission. They don't like to use the term flagship anymore. And we'll talk about the history of that in a minute. But they now refer to them as large strategic missions. And I personally like flagship. I think we should call it that. I think it sounds good. Absolutely. But I think that's really

interesting, too, that you bring up this, the difference in competing. And you mentioned multiple classes in the planetary science line. But of the four NASA science divisions, astrophysics, earth science, planetary science, and heliophysics, they all have kind of a mix of competed and strategic missions and directed missions, center-led missions. Why don't we just introduce this paper first and why we're talking about this. It's called Powering Science, NASA's Large Strategic Science Missions. This is a product of the National Academy's Space Studies Board. This just actually was released in late August. This is a brand-new report looking at the value of these big strategic missions. So actually, Jason, do you want to give a little bit of a background about why we have this report and where it's coming from before we really go

into the details of it with our guest? Being a historian, I have to go all the way back. Let's just talk very quickly about the Space Studies Board. In the beginning. Right, yes. There was nothing. There was a large bang. The Space Studies Board is a standing board of the National Academy of Sciences, Engineering, and Medicine. This dates all the way back to the 1950s, specifically the International Geophysical Year in 1950, I think it was 57, 58. Literally everything in space comes back to International Geophysical Year. IGY, yeah. Yes. Yeah. At the time, it was called the Space Science Board.

The National Academies is a nongovernmental organization of scientific experts that was actually established by Abraham Lincoln. So it's got a very long storied history. For many years, the research arm of the National Academies was the National Research Council. research arm of the National Academies was the National Research Council that was recently disbanded and those functions absorbed into the rest of the National Academies. But the Space Studies Board is a standing board within that structure. And so it is an external board of experts that advises the government. The government contracts with this organization to provide scientific expertise and engineering expertise on various matters. Now, National Academies covers basically anything having to do with science or engineering. The Space Studies Board is very focused on NASA activities, some NOAA activities, occasionally Air Force activities in space or military

activities, but primarily NASA. This particular report was commissioned by NASA, I think in something like, I think the first statement of task was being drafted in about 2012. And what had happened at this time with the Obama administration, they were trying to reduce costs of space exploration. And one of the things that they did in planetary science was to cancel NASA's proposed participation in ESA's ExoMars program, which was originally going to send two rovers and an orbiter to Mars. NASA was going to provide one of the rovers and the launch for this mission. We decided to cancel that. And at the same time, the associate administrator for the science mission directorate, a man at the time named Ed Weiler, retired, I believe, probably due

to some of these financial issues. And they brought in John Grunsfeld, who was a former shuttle astronaut who had worked on the Hubble program. And John Grunsfeld, as he was coming in as the associate administrator, was faced with a speech given by Charlie Bolden, who was at the time the NASA administrator, who declared that the era of flagship missions was over. I have a quote from Charlie Bolden, actually. This is December of 2013. He walked into a meeting of the NASA Advisory Committee of the Science Subcommittee and said, we have to stop thinking about flagship missions. The budget does not support that. That caused big ripples in the

community at that point. Because let's just let's list a few flagship missions. Voyager, Viking, Cassini, Galileo, Hubble. These are the famous missions, the most, I think I can say beloved, also politically beloved. And so this was a, this was a sense shockwaves through the scientific community, thinking that we would lose the ability to fundamentally push forward these big fundamental science questions. And this was from the NASA administrator during, of course, that's broader context, right? This was the sequestration battle. This was broad cuts across the government, but also very targeted cuts coming in at planetary science at this point. So ExoMars was the big loss there.

Planetary science was facing a 20% year-over-single-year cut, which was a pretty devastating cut at the time. As the community was wrestling with this new directive from the NASA administrator, they were really trying to push back on this and say, no, actually, these missions are of incredible value and absolutely necessary for our study of the solar system and astrophysics and heliophysics and on and on. At this point, John Grunsfeld started a statement of task to commission the National Academies, the Space Studies Board, to look at large strategic missions. He was the one who decided not to use the term flagship because he was basically trying to get away from the rhetoric of Charlie Bolden's speech and say, OK, well, these are not flagships. These are just large missions. And what exactly is their value?

We won't do flagships anymore, just strategic missions. Yes, large strategic missions. So again, it's one of those sort of ridiculous government things that happens. And this is part of the reason that Casey and Matt and I all prefer to use the term flagship, just because it's sort of a silly thing not to use it. This report was commissioned. It took a number of years for the statement of task to really come into play. And then these reports take about a year and a half to two years to complete. The National Academies first has to convene a committee of experts, which can take

some time because these are all experts who are volunteering for this committee. And they are, as experts are want to do, have lots of other things that they are actually doing. So you have to find enough people that can get in the room at the right time to convene these reports. And then in my interview with Dr. Ralph McNutt later, we'll discuss exactly how the committee functions and how these reports come about. But that's the background as to why this report happened. And in both of these groups that you've talked about, the Space Studies Board and the group brought together to assemble this report, very, very distinguished people, many of them either involved with a variety of missions, including flagship missions, or certainly might be expected to be involved in

future flagship missions. Yeah, so that's actually an interesting issue that you run across in the Space Studies Board quite frequently and all of their committee reports. The National Academies has a very strong process for resolving conflicts of interest. You know, they're very committed to having an objective opinion on matters of science. But the problem is that particularly in space science, you're talking about fields that are very, very small. So the number of experts that you have to consult, it's not a large pool of people to

draw from. So trying to find somebody who is absolutely not involved in whatever the issue is that you're studying is really difficult. Right. So we have a group of space scientists, almost all who have served at some point on a flagship science mission. And I guarantee you, if a scientist has not been on a flagship mission, they have wanted to be on a flagship mission. Because, I mean, they have large teams, but they have beautiful data that come back. And so that was kind of interesting reading this report. And spoiler alert, this report is pretty pro-flagship missions.

They're very supportive of them. From reading it, I don't detect any serious consideration that they would not be worth it. Can you get an objective view of that from the scientific community that depends on it? Probably not. But at the same time, if you're asking the scientific community, is this worth doing? That's who you want answering your question, right? You don't want just some random schmoes answering this question. Right. But as a taxpayer, I really approve of this process, right? I think if a scientific field is dependent on taxpayer money, I think it's very healthy for them to go through the activity every now and again of justifying why it is and how it is that they spend the money. I like this report because it makes the community think about why exactly they're doing the things that they've always done. As I say, I think that's a really healthy process. And Jason, this is a field, this area of space policy is something that you followed very closely. And it's something that this group talks about in the report and in the recommendations that you'll be talking about with Ralph McNutt. I guess you are one of the world's foremost experts on

trying to track down how big projects like this and smaller ones as well tend to increase in cost and how well or how poorly we do at tracking those increases. Well, I don't know that I'm one of the world's foremost experts, but I'm certainly one of the most persistent pursuers of this information. I think that persistence counts for a lot. Yeah. And just to jump in real quick before Jason expands on that, I mean, what we're talking about here is what they did in this report, which I thought was really nice, was tried to actually, as Jason said, justify or at least quantify some of the returns of these big missions. And so they look at number of publications that use the data from these missions, number of citations of those

publications. Are they relevant to the community? How many scientists are trained on these missions, and what do they go on to do? You had some impressive numbers. Hubble said something around 15,000 scientists have worked on that mission. I thought this was spectacular. The highest number of scientific publications using Hubble data in a year was last year.

You know, that just means like more and more people keep using it. I think it's been cited, their data has been cited 600,000 times. What's actually kind of amazing, and Jason will take over here, is that this is not, actually getting this data for every science mission is actually really hard. And this paper recommends making it easier to find this data so we can share with the public how productive these science missions are. The effort to actually build and launch one of these missions is really staggering. And the people who do this stuff are really good at streamlining their teams and streamlining their efforts to find every dollar that they possibly can to add value to the mission. But as a result, people tend to think only in the immediacy.

They tend not to think about what will happen far in the future or what lessons they can learn from the past missions. And as a result, these missions really tend not to collect a lot of cost and schedule data that they save over time that's comparable from one mission to the next. They're interested in finishing their mission. They're not really interested because there's no reason for them to be in comparing the performance of their mission to other missions.

But that's the kind of data that you really need if you're trying to justify a long-term program of flagship missions or a long-term program of small missions. I have spent years and years and years trying to collect bits and pieces of this data and put it together in such a way that it's comparable from one mission to the next. So I was really gratified to see in this report that the committee came across the same issues I've been dealing with for years and has recommended that NASA try and fix this problem because I think it will benefit the community tremendously to be able to say in no uncertain terms, this is what we spent and this is what the return was. And this is really far more valuable than the financial investment we've put into it.

This is actually what Jason does as a staff at the Planetary Society. We work to gather this data. This is an ongoing project that we have that we're putting together ourselves. And it'd be great if NASA could help us. Before we go into this interview, I actually want to address two points, kind of broad points. And we'll let Ralph kind of really go into the paper itself, which I think is really interesting and really has some really good budget data, really good quantifying data and just kind of arguments for large missions. But let's talk about kind of the usual arguments against them. Why are people hesitant to pursue? I mean, we're talking about these flagship missions. You look at Voyager and you say, oh, of course, we should have done that mission, right? We went to four planets and now we're in interstellar

space, but we had to argue for it. Or Curiosity is a good example. Curiosity went over budget by almost a billion dollars. Can you quantify, is that worth it? If the scientific argument is so clear, why do we have to keep arguing these or what is the hesitation from the budget side or from the government side or whatnot? When you're talking about flagship missions and the history of flagship missions, it's been very rare in NASA's experience that a flagship mission has come in on cost and on schedule. When you're talking about missions that are that expensive, going over budget has far more consequences than a small mission, a Discovery-class mission going over budget. A Discovery-class mission goes over budget, it's probably, you know, if it goes 15% over budget, we're talking about tens of millions

of dollars. If a flagship mission goes over budget by 15%, as you said, with Curiosity, we're talking about a billion dollars or sometimes more. Hubble, all in all, was an $8 or $10 billion flagship. That was amazing. $11.8 billion in terms of all of the upgrades to Hubble over the years. No operations, just development. That was in this paper. Does that include the shuttle launches as well? That they did not include the cost of shuttle launches because there's no real agreed way to quantify how much those cost. But at 11.8, it was the most expensive science

mission of all time is Hubble. Right. So as we were talking about just a few minutes ago about Cassini and the fact that, you know, the past four years, it's cost $200 million in operating costs. Well, if you go a billion dollars over budget on one of these missions, that has serious ramifications for the rest of your community's flight missions and research and development and technology development. So there's a huge risk with these missions. So there's a huge risk with these missions. And NASA has gotten much better at being able to predict the cost and the schedule and work really hard to reduce the scope of the mission so that you're able to get 85% or 90% of the science that you would originally proposed.

But you can do that for half the budget that you would originally proposed. So rather than a $4.5 billion mission, you're sending a $2 billion mission and getting 90% of the science return. So these are the kinds of trade-offs that NASA's really worked hard to be able to predict better so that you don't have these huge cost and schedule overruns. But it's only been in the past six or eight years that a lot of these processes have been implemented. So we haven't really seen the proof that these processes are capable of keeping cost and schedule milestones in check on these large missions. And people are still really hesitant to spend that money,

knowing that there is a risk that they'll have to spend a whole lot more or cancel that mission and just lose all of that sunk cost because it has huge ramifications on the rest of the community. When you say people, particularly we're talking about the actual civil servants who work at the Office of Management and Budget, which approve not only new projects for NASA within the White House executive branch area, but they control the flow of money to these projects. And they're basically the government bookkeepers. And so if they are skeptical about flagship missions that tend to go over budget, it's very hard for NASA to get approval from them to pursue new ones. And I think that was one of the big roadblocks a few years ago when we were really looking at end of a lot of new, particularly planetary missions. And let's talk about maybe the most nefarious recent example of this, which was the James Webb Space Telescope, which was originally kind of sold as. And this paper actually makes an interesting case about NASA needs to be very careful about making promises about budgets before you actually have a formal process of formulation to determine what the actual cost of a mission would be.

But James Webb was sold as about a billion dollar mission. And of course, it was way over that. We're sold as about a billion dollar mission. And of course, it was way over that. We're at a roughly $8 billion mission now. It kind of ate the budget for the astrophysics program. As you were saying, the implications are by so much growth, there's an opportunity cost of other missions to pursue. And you've seen that in astrophysics. They've been working on James Webb since the early 2000s. They have had very few new missions, and the new missions that they've had have been very small, very targeted, very specific, small class missions, because they literally just can't afford anything else. Yeah, that's absolutely right. I think the new start for James Webb was at a billion dollars. It was confirmed, which meant that it went from formulation to development

at, I think, $5.4 billion. And I think the launch date was, was it 2014, 2015, somewhere in there? Yeah, it's earlier. Yeah. Yeah. It would have been launched by now. Then very late in the development process, they found that they were woefully underfunded, massively over budget, and there was no way that they were going to meet their launch

date. So they had to re-baseline this project, which is a process that required congressional approval and they had to do lots of studies and it added another $3 billion to the project and added several years to the development time. And since then- Yeah, the house was so pissed off about it, they tried to cancel the entire mission. Yeah, because it's a huge cost. And as you just pointed out, it was decimating the rest of the astrophysics field. Now, astrophysics is very different than planetary science in that these huge missions that they launch, like Hubble and JWST,

can be used by a large swath of the community. You don't have the internecine battles that you have in planetary science. If you spend $4 billion to go to Saturn, that means that people who want to study Jupiter are just left out of that mission. With Hubble or with JWST, you can use those to look at whatever objects you want to look at. So the community doesn't have that same tension. But at the same time, it means that with Hubble, you're using visible light spectrum. With JWST, you're using infrared. Well, if you're an X-ray astronomer, you're sort of left out in the dark on this one. So literally, yeah. And near infrared by the Hubble as well, of course. And I'm thinking

sometimes it goes the other way where a mission like Voyager, I think Casey, you said earlier, gave us these beautiful views of four planets and their respective systems. But it's now kind of an astrophysics, excuse me, a heliophysics mission as it goes out there past the bubble created by our star. Yeah, and it's not just kind of. It is literally a heliophysics mission. That is the science division that pays the annual operating budget of Voyager is heliophysics at NASA. And it's about, it's actually pretty cheap. It's about 5 million a year. Let me add just one more classical critique of flagship missions is the rate, because they're expensive, it takes a lot of years. You kind of

have to spread the cost over a lot of years to build them. And so you average, let's say, launch a flagship mission once a decade. That does not give a lot of opportunities to build them. And so you average, let's say, launch a flagship mission once a decade. That does not give a lot of opportunities to respond to, let's say, new discoveries in a field, does not give a lot of opportunities to train engineers and scientists on how to run missions. If you're thinking about you want a workforce in the future that's ready to take over. This paper really doesn't address that, honestly. It talks about balance and says we should have balance. And it says then that balance is kind of defined in the decadal surveys by each science division. But fundamentally, it doesn't critique, I think, flagships for that

reason, maybe as much as they should be, because it really is a trade-off. And Jason, we've talked about this before. Science missions will pay for scientists, their salaries. You know, a lot of scientists are either paid part-time by a university or they're completely dependent on contract research grants. Missions themselves will pay some or all of their salaries and their research money. And you can have hundreds of scientists on a flagship mission compared to a small mission, which is maybe dozens of scientists. But at the same time, you don't have training to actually operationally run new missions. And that's a real trade-off. And I think that's why you do, that is one of the real practical values of small missions

is that you're launching, ideally, you're supposed to launch a discovery mission in planetary science every two years. We haven't had that cadence in two decades. You commit to something like Mars 2020, that mission began in 2013, it'll launch in 2020, it'll arrive in 2021. Like that is a long time to focus on a single mission. And there's no real good way around that. No, there's not. And you look at something like Cassini, you know, there have been members on that science team that have basically been there for their entire careers. That's fantastic for them and it gives them a lot of stability and they're wonderful scientists and they, you know, have come up with amazing discoveries using the data from this mission. But that also presents an issue for the younger community. And NASA has been working

to address that in these long-term missions, to find ways to rotate in younger people and give more career opportunities within these long missions. But it's a serious consideration. Well, you know who might have something to say about this? So I take it that we're ready now to go to that conversation, Jason, that you've had with Ralph McNutt, the co-chair of this committee. Their work resulted in this report, Powering Science, NASA's Large Strategic Science Missions, otherwise known as flagships. I just want to mention one thing that I found really touching and very appropriate, that the committee dedicated this report to the great Neil Gerrills, the astrophysicist from Goddard Space Flight Center, ran the SWIFT mission,

was helping to shepherd the James Webb Space Telescope toward space, and passed away very untimely earlier this year. And so I thought that was a very appropriate move on their part. Yeah, I agree entirely. We're seeing sort of a generation of planetary scientists and astrophysicists who are nearing the end of their careers. And these folks have contributed a tremendous amount to the United States space science effort. I agree with you. I'm very happy that they mentioned that in this report. And I think we should honor these people in any way that we can. Today I will be talking with Dr. Ralph McNutt who is a researcher at the Johns Hopkins University Applied Physics

Laboratory. He's been there for many years and has worked on a whole host of planetary science missions over the years. He was on Voyager, he was on the Cassini mission, he was on the New Horizons mission to Pluto. At APL specifically, APL has run a number of planetary missions themselves, including the Messenger mission, which Ralph served as a co-investigator on. He's currently also working on the Parker Solar Probe, which will launch next year, which is also an APL mission. Ralph has also served for many years on various, now the National Academies of Science, Engineering and Medicine. many years on various, now the National Academies of Science, Engineering, and Medicine. He's done chair work and membership work on several committees, most recently on their project. He's the co-chair looking at NASA's large science missions. We are very pleased to have you here, Dr. McNutt. Thanks for coming on. Well, I'm glad to be here, and I got tired just listening to you.

on. Well, I'm glad to be here, and I got tired just listening to you. Well, it's an impressive resume, and I like to highlight that kind of stuff. People often are unaware that scientists work on numerous missions over the course of their career, and it's an impressive list that you've got there. As I tell everybody, you know, it's okay, though, because they're all underfunded. That's right. That's right. We're working hard to see if we can't change that. I'm sorry, Jason. I'm going to get myself into trouble really rapidly here. Well, if there's anything that you say that you decide you don't want on, just let us know and we can certainly cut it out. That's okay. I was wondering if you might be able to tell us a little bit about the process by which you end up as a co-chair for one of these National Academies

reports. It's an interesting process, I think. A lot of people don't understand that this is a voluntary role. Well, it is a voluntary role. One ends up being invited by the academies that are running these studies to participate. They talk a lot amongst themselves, the people that are setting the studies up. I don't even remember which one of these things that I was originally involved with. I was involved with some at some lower level over a decade ago. I don't know. Apparently they liked my contributions enough that I keep getting invited back. enough that, you know, I keep getting invited back. I've thought that this, you know, these sorts of things are important enough that I have accepted doing them. The last really big thing

that I was involved with, well, the last two things, I was the co-chair on the radioisotope power system study that came out in 2009. And I was also on the steering group for the that came out in 2009, and I was also on the steering group for the Planetary Decadal that I believe was finally published in 2013. So I don't know myself exactly how one ends up getting invitations. People get recommended, and then it's a question of, I guess, whether or not that they like what you did in the past, and so you can end up being invited back. So far, they invited me back to help out with this one, and I was glad to do that. And we'll see if I get invited back for another one in the future. Okay, so that's interesting in and of itself.

I was wondering if you might be able to walk us through the process of writing the report just a little bit. What kinds of meetings did you set up? Who were the people that you talked to? How you guys decided to divvy up the writing assignments? So one of the things that's really critical with all of this is the support of the staff at the academy. Wayne Day was the study coordinator on this and worked with both myself and with Kathy Thornton and really, really did a lot of the legwork on coming up with suggestions on who it might be appropriate for us to invite within the Academy, helped to go ahead and expedite getting those meetings set up. We actually ended up having three different meetings. So all of this sort of got started, I think the initial inquiries were about in April of last year.

In April of 2016. So the first actual meeting we had was the 5th and the 6th of October in 2016. That was in Washington. We talked to the new NASA Associate Administrator for the Science Mission Directorate. That's Thomas Zurbuchen. I think Thomas had been on the job for about two days when we had asked to have a chance to talk with him, and he was extremely helpful. We also talked with all four of the division directors. We talked with the former AA, John Grunsfeld, who was actually the person at NASA that actually asked the academies to do this study. And we also talked to some of the current and former congressional staff.

The second meeting we had was on the 5th through the 7th of December. That was out in Irvine, California at the Beckman Center, one of the academy's facilities. We talked with former chairs of decadal surveys. We talked with principal investigators for large and small missions. And we also talked with experts on cost estimation because this was a big piece of this study as well. And then we had a third meeting on the 15th to the 17th of February. That was in Washington. And that was really mostly writing, following up with NASA sponsors for revised input.

One of the things that we had done was to ask to get actually quite a bit of costing and scheduling information for missions that were, that had been accomplished by the Science Mission Directorate over the last several years, as well as getting a good snapshot of what was going on currently. And Dr. Zurbuchen and his staff really pulled out all the stops in terms of helping with that. And it was really a huge help, and it was also a huge effort, because a lot of what actually ends up getting done within the science mission directorate really is done within the divisions. And I think one of the things that we recognize in the course of doing this study is that there's some, I would call them perhaps

interesting sociological differences between how the divisions operate. And really that's rooted in what the science is, how those science communities interact with themselves and with the exterior world, and also a lot of historically how the funding has flowed in the past. I mean, there were originally astronomy and physics lines that were set up within NASA in the first NASA budget in 1959. And, you know, all of this really goes in terms of the science all really goes back to the enabling legislation that set up NASA back in October of 1958. So we really tried to do a great deal of due diligence in looking through all of this and tried to talk to really a wide variety of people. We were trying to really get input from all the various stakeholders who have various points of view on things, perhaps in some cases because of the fact that these are people that are actually running the divisions, people from the outside, like I say, former congressional staff,

you know, were very knowledgeable about the NASA budget. We really try to take a very broad brush approach to all of this. Yeah, I certainly know from my own research that trying to find budgetary and scheduling numbers for particularly historic NASA projects is really an interesting exercise. And I give you guys a lot of credit for having been able to work with NASA as closely as you were to get some of that data. And I know that that actually comes out as one of the recommendations, and we'll get to that in a little bit. Right, right. One of the things that I found really interesting about this particular report, it's trying to compare, as you mentioned, the differences between the various science mission directorates and the divisions within SMD, the science mission directorate at NASA. It's really sort of an apples to oranges comparison very often. So trying to

figure out how to talk about this in a meaningful way, I thought was really interesting. And I was And I was wondering if you – just from the very basic definition of a large mission at NASA, I was wondering if you might be able to elaborate a bit on some of the issues you guys ran into with that. Well, actually, it was even worse than what you described. This really was at the heart of the charge that we had been given, you know, within the statement of tasks that we had. In particular, I think the statement said, they asked, what are the general principles that SMD could use to trade off within a limited budget between development and operation of large strategic missions in the cadence and our cost caps of medium size and small PII-led mission lines? Okay, sounds like a really, really simple question. But as you know, there are apples and oranges comparisons, and sometimes we were not quite

sure whether we were dealing with apples and oranges or kumquats and pomegranates. But what NASA does is really complicated. And the science is out there right on the cutting edge, as it should be. The technology for doing these missions is right out there on the cutting edge, as it should be. It's complicated. This was all framed in terms of large strategic missions. And talking with Dr. Grunsfeld, part of the motivation for this, quite frankly, was the overruns that had been incurred by agency about whether it was really appropriate to keep doing large missions because if they overran a little bit in terms of percentage, it was a great deal of money because these things cost a lot to begin with. So as you say, one of the things that we were trying to figure out to start with was, well, okay, within the individual divisions, exactly what does one mean by a large mission? I mean, sometimes these things have been called flagship missions in the past. That's a term that

has had some bad connotations that has been linked with it. And so sort of the nomenclature that had been running within NASA was this idea of defining a large strategic mission. We were trying to kind of figure out what exactly that meant. One of the things that we concluded early on was that you can have strategic missions, that is, a mission that needs to be done within the context of the decadal surveys from the different divisions. Just because a mission is strategic, typically it means it's a mission that is directed to one of the NASA centers or a de facto center. Not all strategic missions are necessarily large, but all the large missions really are strategic.

So that was, I think, one of the important points. Then it's a question of kind of looking at the costs that are associated with these. And certainly, if one looks back at what things have cost, if one looks back at what things have cost, the really large missions, the largest ones, have tended to be over within the astrophysics division. And this is just really sort of the nature of the beast. I mean, one of the things that Dr. Zabrokin was very helpful about doing was providing a figure, which is reproduced in the report, that looks at 31 large strategic science missions across 57 years. And those numbers are directly from NASA.

And this is one of the things that had actually been put together for an earlier Academy's report. And it's rather interesting. I mean, if you look back, this is going, I think the figure there goes back to 1976 and runs out through the early 2020s. And FY15 dollars, there's about, it's about $70 billion, which is a lot of money. And again, this is normalized to FY15 dollars. There's 31 missions that are involved with that. It turns out in those corrected numbers, the most expensive mission that's been flown has been Hubble. And Hubble came in at a little over $11 billion in those numbers. And of course, that's and counting because we've still got operations going on with Hubble. But for almost all these missions, the majority of the cost

really are getting the things up through commissioning right after the launch. It turns out the other things that were sort of interesting was under this sort of rubric, the most expensive mission that the Planetary Science Division has flown in that same area is still the Viking missions. That came in at a little under $7 billion, and again, in FY15 adjusted dollars. The Terra mission and the Earth sciences division was a leader at about $2 billion, and the magnetospheric multiscale mission within heliophysics is the largest mission to date that has been flown within heliophysics. And it's just over 1 billion. The Parker Solar Probe, as you mentioned earlier, which I've got an instrument on, our launch window opens on the 31st of July of next year.

And the number for that's a little over 1.5 billion. But the interesting thing is that if one actually looks at, for example, what's considered a large mission within planetary, it's over $2 billion, whereas if you look at heliophysics or if you look at earth sciences, it's something over $1 billion. So there actually are these differences. It's not like that you can say, what's a, you know, what's a really big mission, a large strategic mission within the science mission directorate. It's something that you end up, you know, really having to talk about, well, which one of the divisions is this,

is this within, what some of the functionality is this within, what some of the functionality is, and what are some of the issues. Like everything else it's a bit complicated and it is one thing that I think that we did a pretty good job of at least getting our hands around because you know this sort of distinction is something that I think within the Science Mission Directorate and indeed within the divisions, it's something that everyone sees as being fairly obvious. But I'm not that convinced that it is so obvious to people that are on the outside looking in. So hopefully one of the things that we've been able to do in the report is to provide a little bit more by way of definition and by way of these examples as to what some of these inherent differences are. And again, they trace back to the kinds of science that are being

done and simply what it takes to be able to do those and indeed how the science community is structured that's making use of these missions to be able to push our knowledge base forward. That's absolutely a tremendous value I think this report provides. I'm very happy to see it. I think this will be a very useful tool for NASA going forward in trying to promote some of these missions on the Hill and at OMB. So I think this was really a valuable service. I was wondering if you might actually run through some of the recommendations that the committee came up with. Sure. There were sort of varied groupings of the recommendations. One of the things that I think we came down pretty hard on is the idea that one of the things that you need to keep in mind is that we were kind of

asked originally, is there sort of some general principles that NASA can apply to figuring out how to deal with missions? And I think that the thing that we all ended up feeling very strongly about is that this is something that really is within the purview of the various decadal surveys, which of course are done division by division. They don't all happen at once. They're staggered in time, but that it really is, you know, it's having these decadal surveys, which enables all of the members of the scientific community for that, that, you know, relate to that particular division to come together, that it really is within their purview of trying to determine

what the priorities are, what the appropriate balance is, how one should go forward. Now, in doing that, we said, well, one of the things that perhaps should be looked at by the large strategic mission proposal teams is to really try to be thinking through, again, this is when the decadal surveys are going on, as well as some of the homework up front, to be thinking about minimum science goals, maximum budgets, and really trying to get a good idea of what various science goals might be desirable at different budget levels. Because one of the things that one has to worry about, of course, is how much money really is available. And to the extent that the scientific community and NASA can develop less expensive implementation strategies for mission concepts that do not

exceed the current budget limitations, you know, that's a win-win situation for everyone. So that was one of the recommendations. I thought that was a really interesting recommendation. And the reason for this is if you look at the smaller mission programs, your New Frontiers, your Discovery program, those proposals are prepared by principal investigators, and they're basically doing those proposals on their own dime. So there tends to be a concern about adding requirements to the proposal process because it's adding burden to the principal investigator teams.

This being a strategic mission, I think it's really interesting to add these extra requirements in the proposal because they're not under those same constraints. You know, it will cost NASA a little bit of extra money, but in the long run, it ends up saving NASA a tremendous amount of money if you can find better ways to do a certain percentage of the science for a far less cost, right? So I really approve of that recommendation. I thought it was- Yeah, well, you know, and again, this is really part of the thoughts that we had as well when, you know, when trying to look at this. And sort of going along with that, we had said, well, that the decadal survey should really be looking at

mission concept variants or other means to assess, you know, what are the boundaries of cost and technical risk that makes sense? Now, certainly one of the things about the large strategic missions is that you are trying to certainly be pushing the boundaries of what you can do with the science and technology. I mean, if you're going to spend all of this money, you had best have a good reason for doing it. And there are certain things that are out there that the decadals, for example, have identified as being important where you really can't do the science. You really can't do the implementation any other way. It's inherently going to be expensive because it's going to be big. If you look at Hubble, if you look at Chandra, if you look at Curiosity, if you look at Terra, Aqua, and Aura within the Earth sciences, if you look at magnetospheric multiscale, if you look at the top of the game of what one needed to do scientifically in order to do the measurements that were needed to really move things forward. You need big sensors. You need to be able to go to places like with Parker Solar Probe where they're not easy to get to within the outer corona of the sun. And so these are not going to be cheap things to do.

I mean, that's just sort of the nature of all of it. The point is, is that that said, it is, again, it's a win-win situation to everyone's advantage to be able to understand what sort of cost and technical risk and trade-offs that there might be. And, you know, this is something very hard to do at the decadal stage because we're not talking about a mission that's been through a full phase A and a full up cost estimate necessarily. But you do need to be able to have some input in that, and there's some other recommendations on that, so that really a responsibility for the decadal surveys is to be able to come up with some decision rules that can provide some flexibility back to NASA management so that budgets go down and

one has to downscale the program, well then, you know, there's a way of doing that. At the same time, although, you know, history hasn't been with us too well on this recently, but one could end up with more money than one had been planning at the time of the decadal, for example, halfway through the decade. In case that happens, you know, you need to have some ideas of what possibly could be done to upscale the program. You know, we all dream of winning the lottery, and occasionally somebody does. And it's sort of within the purview of the decadals to really be thinking all this through. Because again, we have these midterm assessments, but the decadals per se only meet once a decade. That's why they're called decadals.

And so it's a significant amount of time over which there can be new discoveries, over which there can be and there will be changes in the political landscape. Certainly looking back at, well, if you look back at all of NASA's history, I mean, predictions of budgets over a 10-year basis have tended to vary quite a bit against what the realities were. So it's important because the stakes are so high that some thought be given to all of that. So another one of the recommendations with respect to prioritization, one of the things that NASA needs to have some robust mission studies that can allow for some trade-offs to be discussed on potential

large strategic missions, and to do that prior to the start of the decadal survey. Now, again, I mentioned I was involved with the steering group on the last planetary decadal, and one of the things that did happen, Jim Green provided study money to do some fairly robust studies of technically what could be done on a large variety of missions. And all of this was happening in parallel with the decadal itself,

because people had notions that they came up with. And, you know, we were trying to get some sort of an assessment. And at the same time, the Aerospace Corporation had been, again, on planetary, had been under contract to, actually to the academies. And Steve Squires had a, and the academies had a – Steve, of course, was the chair – had a large role in really looking at a variety of organizations that could do some cost modeling. the CATE process, which is not really meant to be a cost estimate so much as trying to look at what they call the cost. CATE stands for cost and technical evaluation, to at least be able to intercompare on some roughly similar footing the various missions and concepts within the division that are being considered. It was a little bit of the Wild West because, again, there was a lot of pressure to

really be doing the technical studies as well as the CATE studies real time at the same time as the decadal because you were really looking at notions that were coming up at that time and somehow everything had to complete at once. So in a way, it was not that dissimilar from, you know, on a much different scale where that people, for example, within the Explorer program would actually, you know, be preparing a proposal. within the Explorer program would actually, you know, be preparing a proposal. In a way, it really was the same kind of pieces of things, except that it was the whole community that was basically trying to come up with some evaluation on a proposal

with which to go forward for whatever the particular NASA division that was being dealt with and to do it over the next 10 years and somehow make sure that as much of that as possible could actually happen. And of course, we've got the midterm assessments that go on to take a look and see how well we do. And we've got the past decadals now for all the divisions. And so a lot of this continues to be a learning process as well. Yeah, that was a point that I highlighted earlier in our show, that the decadal process is relatively new for most of the divisions. But I think it's been a really powerful tool. I think it has worked tremendously for all of the divisions in being able to assess their priorities and then accomplish those priorities. I think it's a wonderful messaging tool for outside stakeholders.

But I think one of the most impressive parts of this process is that it is so adaptable, that you can learn as you go and make it better. And I think that's really impressive. Right. In order to do that, you know, people have got to stand up and be responsible and work together as a community. And at least I've been involved directly with both the planetary as well as the heliophysics decadal. I was on one of the sub-panels on that.

And I think that these communities are very responsible. And people do take this seriously. And, of course, that's an important part of making all of this work. And also being open to going back and looking and seeing at, you know, how well things have worked in the past and what things have not worked. This is all a moving target. I mean, things do not stay static. And the way that NASA operates, the way that budgets go, the technology, the twists and turns of the science, you've got to be a little bit light on your feet, but at the same time trying to figure out how to plan for the next 10 years when these things come up. Yeah, on budget estimations, I've already really talked about a lot of this, up. Yeah, on budget estimations, I've already really talked about a lot of this, that, you know, that we concluded that the budget constraints really should be included in the development of the decadal scientific program. You have to have flexibility. I mentioned the upscoping.

We all need to be optimistic every so often. And then also that the, you know, that this is not an ironclad thing. I mean, the other side of this is that you're using this kind of information to inform what should go into the decadal surveys, but not to narrowly restrict what's going in. Because future projections of available budgets are future projections, and those can go one way or the other. budgets are future projections, and those can go one way or the other. And again, you have to worry about the flexibility problem and have that built into the decision rules. One of the other things that we talked about was that it really is important for NASA to keep using the various cost estimation and cost management tools to assess and control the cost and risk. Actually, if you look back over the last several years, NASA is doing a much better job. There's a set of GAO, Government Accountability Office reports. Typically, these come out in March of a given year. I think the one for 2017

actually came out in May. It was delayed because of the election. But actually, these have been coming out on a yearly basis since 2009, and the GAO has been tracking quite carefully how NASA has been doing. And we refer back to these reports within our report. And they do show that NASA is actually doing better. I mean, we're not exactly – NASA is not hitting everything exactly on the mark, but the trends are certainly improving, and NASA is doing a better job. One has to keep worrying about supporting the development of new tools to do robust cost estimates and risk assessment. I mean, this is, again, a bit of a moving target.

Things are changing. People's perceptions change. People's perception of risk change. And one has to somehow be able to keep up with that. And to the extent that these new cost estimation tools are being developed, they're also something that can support the decadal surveys, which is another important piece. support the decadal surveys, which is another important piece. Looking through here, I think the last one of our recommendations, which is perhaps a little bit controversial, is that we really said that NASA probably can publicly make

a better case than they have been on a lot of this stuff. These large strategic missions, they are incredibly scientifically productive. They have been incredible means of helping to push forward the development of various space technologies. They've been incredibly helpful in educating new cadres of graduate students. And basically, they're an investment in the, in the future. One of the things that we actually sat down and all tried individually to do was to, and again, this got back to the charge about looking for, looking for metrics and assessments. Of course, one of the things that, that is very commonly put is people say, well, you know, what you ought to be able to do is maximize science. And that means you look at the science output and you divide that by the cost. And so

that should be simple enough, except for the fact that when you look at the cost, you've already got to be a little bit careful about that, about again, making sure that you're comparing apples with apples and not kumquats with pomegranates, as I said earlier. It's even more problematic to assess the science because there's sort of two pieces of that. I mean, there are bibliometric tools that are out there that people, and of course, the most famous or infamous one is the H-index that Hearst came up with in about a decade ago. People look at individual people's scientific productivity, and that has spawned at least one peer-reviewed journal that I know of.

I think there may be two, and an incredible number of arguments in the literature and outside of the literature. One can do that sort of same methodology and try to apply that to a mission. But of course, even if you could get rid of all of the uncertainties and problematic parts of trying to do that kind of an analysis, which you can't inherently, and we again had a lot of discussion about this, inherently. And we, again, had a lot of discussion about this. You're looking at something that you, you know, you have to have a mission that's been operating for maybe 10 to 20 years before you can really come up with some sort of an assessment of, well, you know, how much has this really,

you know, shaken the, you know, the scientific establishment of change paradigms. It's not something that you're going to figure out prior to the launch of a mission. No, it's inherently a lagging indicator. Yeah. It is inherently a lagging indicator. But nonetheless, the idea that, I mean, one of the things that happens with NASA right now is this whole idea of trying to do extended missions. There's extended mission proposals. And one of the things that has happened with those is that typically people have been gathering things like citation data. Some of this stuff is out in the public. Some of this stuff is on websites.

Some of it's not. It's in all sorts of different forms. And so it makes it very hard to intercompare. It is interesting, I think, that if you look at the number of scientific peer-reviewed papers and the number of citations of those papers, that the largest number for anything is for Hubble, papers, that the largest number for anything is for Hubble, which is kind of good since Hubble was the most expensive of the missions that we've, you know, we've flown in the last, in the last, you know, many decades. So at least qualitatively, that all kind of hangs together. And quite frankly, I mean, one of the things that has happened, not only on the astrophysics side, but also within the telescope community as well,

is that there have been periodically a variety of publications basically making the case of what kind of scientific productivity that there has been from the telescopes, from Hubble, from Chandra, just to name a few. Also on Earth sciences, that's been done. Not so much in planetary and in heliophysics. I think part of that has to do with how that the budgets are structured. But nonetheless, these are kinds of metrics that NASA, certainly via the extended mission proposals, could start asking people for in a common format. And it would be a little bit of an experiment in doing all of this because people, again, people see the way that the scientific stories and the scientific productivity unfold are different. But there's at least interest by some of the stakeholders out there,

are different, but there's at least interest by some of the stakeholders out there, and typically the ones that control the money. So they're important as a result. That, you know, at least having this information available, along with as many footnotes as you want to on the data, we thought would not be a bad thing. It would be an experiment to run, but we thought that it would be a worthwhile one to run. Trying to backfill data of that nature is really hard to do, but certainly in going forward, this is something that, you know, at least one could think about trying to do across the entire science mission directorate, but, you know, it's effort. And if somebody is going to be paid to do that and paid to do that coordination, and it's not that, not totally that trivial, when there's something else, it's not going to be done. Again, it's a recommendation. It's back to NASA management and, you know, they're going to have to decide within NASA whether that it would be worthwhile doing or not. It was certainly our thought that transparency is good and being able to get out in front and really try to let people know, the American taxpayer,

try to let the American taxpayer know what they're getting for their money. And I think it was interesting looking across the various divisions that perceptions are different. And by being able to at least try to, you know, trying to do that in some sort of a unified fashion, it might actually be, it might actually be helpful, you know, with within NASA as well. The various division directors might have, and the various division personnel might have a better feel for kind of where they fit in the grand scheme of things, or have better ideas of how to better characterize what that they're doing, and to do a better job. And doing a better job in explaining what we do to the public, I think, is not a bad thing.

As a historian, researcher, and analyst, I am all in favor of collecting more data. So I totally agree with that recommendation. The caveat is you do have to be careful, and the footnotes do matter. Yes, absolutely. That is absolutely the case. Well, Dr. McNutt, I really appreciate your time. Unfortunately, I think this lively conversation has taken up all of our time. Just want to tell you, I really appreciated the work that you guys did on this report.

I think it's a tremendous report. It's very interesting reading. I highly recommend that our listeners jump online, and we'll link to it on our website. And I hope everybody's able to check it out. And thank you so much, again, for your time. I really appreciate it. Well, thank you for having me. And again, hopefully we've been able to make a solid contribution. I think we have, but we'll see. It always takes time to kind of see how these things cascade through the system. Absolutely. Thank you so much.

You bet. how these things cascade through the system. Absolutely. Thank you so much. You bet. Jason Callahan, Space Policy Advisor to the Planetary Society, in his conversation with Ralph McNutt, the co-chair of the Committee on Large Strategic NASA Science Missions, Science Value and Role in a Balanced Portfolio. That's the official name of the committee that delivered this report just a few weeks ago.

The report is called Powering Science, NASA's Large Strategic Science Missions. And we will put up a link to that report, including the summary. It's 133 pages, but the summary is much easier going, easy on the eyes, and has all of the recommendations that Ralph and Jason were just talking about. We will put that up on the show page that you can reach from planetary.org slash radio. Gentlemen, any closing thoughts? Jason, that was a great interview. This is one of those areas where, you know, we have to just respect that reports, even if they're read by a relatively small number of people people are still important.

And I think why we do these things and why we have the Space Studies Board and the National Academies is that it does represent a summation of thinking by a large segment of the community that goes through a very formalized process. So you and I have put out reports ourselves. This was a lot of work to put this together

and really interesting insights coming out reports ourselves. This was a lot of work to put this together and really interesting insights coming out of this. So just glad to see this type of reports coming out from the community and from NASA. And I would just like to highlight that this report is indicative of a process, a very hard-won process from within the scientific community. It's taken them 50 years to get to the point where they can come together, have these discussions internally, make all the sausage, and then come out with a very unified voice. That's a very powerful tool when you're talking to policymakers, when you're talking to people on the Hill, talking to people at NASA. If you can point to a single report that has the backing of the entire scientific community and says, this is what we believe, that's a really powerful tool.

And I suppose the topper of all of those is, and it's a ringing endorsement of decadal surveys out of this report, as you pointed out during, and Ralph McNutt pointed out during your conversation, the decadal survey is sort of the ultimate example of respect for that opinion of the scientific community. That's absolutely right. And for planetary science, they have only had two of these so far. So this is still a fairly young process, even though it's taken place over 20 years. But we're talking about flagship missions that can take 15 or 20 years to come to fruition. So this is, it's a young process that seems to be very poignant, very powerful, but it's something that I think the community is going to have to fight for continuously.

Guys, it has been a blast. Once again, educational, inspiring. I think we ought to do it again. Okay, well, that's a good idea. Maybe, let's say, next month, same time? Yeah, how about Friday, October 6th, since we shoot for the first Friday of each month for the Space Policy Edition of Planetary Radio.

That's pretty much it for this one, the September 2017 issue. Number 16, we'll go on to number 17 next time. A reminder that we would love to have you join us in helping the Planetary Society to conduct this kind of work, gaining even more respect and support for planetary science and

all of space science by becoming a member at planetary.org membership. Please consider that. Check us out at planetary.org. We'll be back with you on October 6th for the next Space Policy Edition. In the meantime, of course, the weekly edition of Planetary Radio. There's a new episode posted online every Wednesday, and we're looking forward to saying goodbye to that flagship mission at Saturn, Cassini. We're going to have a big celebration, a live celebration,

with KPCC, Southern California Public Radio, on the evening of September 18th, just three days after that flagship spacecraft plunges into the atmosphere of Saturn. Hope you'll be able to join us for the live stream of that show. And guys, I will see you again in October. Absolutely, Matt. And let me plug one thing. If you become a member by the time you hear this, the next issue of Planetary Report, our magazine, is going to be all Cassini. It should be

a really beautiful issue. And if you sign up now, you'll get it just in time for the end of the Cassini mission. You want that. It's a great quarterly magazine. Still one of the most prized benefits of being a Planetary Society member. All right, Jason, I want you to take us out with that theme song that you came up with for us.

Have a great month, clear skies, and we will see you in October unless you join us for the regular Planetary Radio program next week.