Episode 17: Navigating Uncertainty

TESS VIGELAND: We’ve been driving through fog since the beginning of this pandemic. We still know so little about how coronavirus works, how infectious is it, how deadly. What can we do to slow it down?

EDUARDO PORTER: Can economics help us make decisions? Particularly complex high stakes decisions about closing down businesses or opening schools when we have such a shaky grasp of the consequences? 

TESS VIGELAND: This is Pandemic Economics, a podcast about the global impact of COVID-19 from Stitcher and the Becker Friedman Institute for Economics. I’m Tess Vigeland. 

EDUARDO PORTER: And I’m Eduardo Porter. We’ve been invited to have conversations with University of Chicago economists. In this episode, I’ll talk with Lars Peter Hansen, professor of economics of the Department of Economics and the Booth School, and recipient of the Nobel Prize in economics. And joining us for this conversation is Constantine Yannelis of the Booth School of Business. We spoke about the problem of uncertainty and what can be done about it. 

So the messaging about coronavirus has been all over the map. 

TESS VIGELAND: Yeah. 

EDUARDO PORTER: Remember when you know masks were supposed to be pointless, we were supposed to leave them to the medical professionals. And now it seems that masks are indispensable. 

TESS VIGELAND: Yeah, absolutely. We all thought that only people with symptoms were contagious. Now we’ve discovered that’s not quite true. Epidemiologists didn’t think that airborne transmission was a big deal. Remember, we were all scrubbing our surfaces with Clorox and now we know it’s a little different. You know, I think in a pandemic like this where it’s all brand new, maybe that’s inevitable, right? Everyone, policymakers, the rest of us, economists– everyone dealing with this unknown pathogen. 

EDUARDO PORTER: You know, policymakers are stuck in this real fix where they have to make decisions about closing theaters and restaurants and schools, relying on models about how many people are going to be infected and how many people are going to die that are not really a solid representation of reality. There’s enormous uncertainty about which model is going to be right. And they nonetheless have to make a call. 

TESS VIGELAND: So how do they do that? Perhaps economics can help. Lars Peter Hansen has been thinking about this– how you make decisions when things are so uncertain. 

EDUARDO PORTER: His model is really complex. But at the end of the day, the final output is kind of like a set of rules or mechanisms that will help policymakers kind of navigate the uncertainty, right? Incorporating information from all the models that are out there to design policies that can understand that there are scenarios that you want to be able to deal with if they actually come to pass. 

TESS VIGELAND: All right, so you spoke with Lars and with his colleague, Constantine, who I actually spoke with in an earlier episode about the pandemic’s impact on consumer decision making. And you’re talking about this whole issue of uncertainty and how policymakers or anyone, really, usually respond and how they might take a second look at that approach. 

EDUARDO PORTER: Now you both are versed in a field of economics that could provide some pretty invaluable assistance, I think, to policymakers– this decision theory– how to make decisions under conditions of uncertainty when you don’t know for sure the true nature of the underlying reality. Lars, how could you step in and say, help the poor governor in, I don’t know, Florida or Texas or California, who must decide whether to shut down the state economy to prevent the new wave of contagion? I mean, what kind of rules could be thought of and deployed to give some coherence to policy? 

LARS PETER HANSEN: Absolutely. There are some interesting principles. And I think here’s where decision theory under uncertainty kind of adds some insights that are not commonly explored. So if you watch the pandemic, you see lots of model builders– epidemiological models where they make these different projections and forecasts and the like. And the protocols where the kind of aims of those forecasts can sometimes be different. 

Sometimes, the aim is to give you what’s your best guess is what’s going to happen in the future. And so you use your model to make that guess. The first thing I would say is in a state of uncertainty, borrowing something from St. Thomas Aquinas, beware the person of one book, beware the person of one model. Because you want to be looking across different models they each have their virtues and liabilities. 

So there’s that uncertainty. And then the models themselves, if they’re not quite right, they’re approximations, they may know this stuff as well. So you make projections off a model. They can be best guesses. But sometimes, they’re warnings. You can say, well, the good model not only tells you our best guess, it also tells you what bad could happen. 

And so part of what decision theory wants to do is help you think about that trade off between what’s your best guess is what’s going to happen versus what adverse consequences could play out. And decision making under uncertainty involves that trade off. And that’s really where these type of tools come into play. 

EDUARDO PORTER: We’ve been battling COVID in the US since, what, March. So we’ve got a track record of decisions across the country over the last four months. Governments have been doing things in response to what they see coming at them from models and other sources of information. So Constantine, how would you grade the responses so far? 

CONSTANTINE YANNELIS: So I think the first principle that a policymaker has to take in this kind of situation is to recognize that there is a lot that they don’t know and that it’s impossible to know in the short term. So Lars mentioned Thomas Aquinas. When I think of this, I often think of Plato and Socrates. In The Apology, Plato writes that Socrates is the wisest man in Athens because he admits that he doesn’t know many things. So we have to think about explicitly what we don’t know, how to model that, and how that affects our decision rules. 

And unfortunately, in many situations policymakers took the opposite approach of what decision theory and economic theory would tell them to do. For example, there’s a quote from Bill de Blasio discussing whether New York should cancel the St. Patrick’s Day parade, and he says, well, we don’t even know what the true impact of this virus is, so I’m not sure it makes sense to cancel a parade if we don’t really know how bad this virus is. And his lawyers mentioned there are a wide variety of models, especially early on. So policymakers had to make decisions not knowing the true health and economic consequences of the virus. 

EDUARDO PORTER: It seemed to me here that the answer in de Blasio’s quote that you shared with us a moment ago is given that we don’t know, let’s not act, which strikes me as an odd choice. 

CONSTANTINE YANNELIS: Exactly, which is not what economic theory tells us we should do, but that unfortunately has been the reaction of some policymakers. 

LARS PETER HANSEN: Yeah. when you talk about uncertainty, that’s one danger when you start mentioning the term– is that people can easily abuse it. 

CONSTANTINE YANNELIS: Yeah. 

LARS PETER HANSEN: Since we’re not uncertain, let’s go find the outcome we like and go with that. Or as you say, do nothing in the sense of, well, let’s just ignore the problem since we don’t have a precise answer to the possibilities. And what you ought to be thinking through is what the range of possibilities are and how bad they could be. But you don’t have to know for sure something bad’s going to happen. Just the possibility that something bad could happen should be enough for you to act now. 

CONSTANTINE YANNELIS: Another problem with policymakers’ decision rules is as Lars mentioned, they often put too much emphasis on one model or taking an average model or revision of the model. Whereas in truth we know that likely, all of the models are wrong to some extent. They’re approximations of reality. And policymakers need to think about and consider the worst possible outcomes, even if their best guess follows one model or a weighted average of different models. 

LARS PETER HANSEN: Let me qualify a little bit this worst case. If we don’t bound possibilities out of things, then the worst case could be don’t get out of bed in the morning, because everything is  hopeless. And so part of the trick of this is not really going to some very extreme worst case, but to some– and again, bound the different possibilities that might happen, either probabilistically or in some other ways. And that’s a key part of making smart decisions– is to kind of sort out where those boundaries are. 

I always get nervous about this reference to the worst case in the sense of part of sensible decision making doesn’t mean that we kind of have five different models, we have no idea to weight them, and therefore we should just figure out the one that has the worst consequences, and implicitly put all the weight on that. I think that’s too extreme of a way to go. 

For me, I think a more sensible thing to be doing it is imagine putting different weights across a different models and looking at sensitivity as it changes a little bit, but not necessarily always going to the extreme. But I think the most important point is this trade off between what you think is likely to happen and what could possibly happen. And as implementing decision theory, I can help you think about that trade but I can’t tell you how to make it. In the context of COVID, the real problem is the fact that things have been unraveling very quickly, and it’s been very hard for them to respond instantaneously to all the information that’s been flowing. 

EDUARDO PORTER: If you look out at the world, you don’t see changes in policy that come from the new information. Now pretty much everybody has a mask ordinance, which, like, a month ago was not happening. So that seems to be a result of some narrowing of uncertainty about the use of masks. But I’m thinking, is there a way to think about how to use this new information wisely, Constantine? I mean, how should the decisions evolve and how should our methodology to assess the different models evolve as new information comes in? 

CONSTANTINE YANNELIS: Well, as new information comes in, I think there are two obvious ways in which we can adapt the process. One is simply having greater certainty about the parameters to feed into the model. And two, we can alter our weights regarding which model is correct. For example, I believe early on in the crisis, models predicted 100,000 deaths in the US. We can emphatically reject those now, given the data that we’ve observed. 

LARS PETER HANSEN: Yeah, there’s one point here, which I would like to add about part of the real challenge here is epidemiologists have their favorite models. They have their models and they’re built on epidemiological expertise. Economists know less about epidemiology, but know a little bit more about some of the economic trade offs we face and the like. 

To do this sensibly requires what some people might call an integrated model or integrated assessment model. It involves inputs from all sides, because at the end of the day, the economic behavior impacts the behavior of the pandemic. And the pandemic itself affects economic outcomes. And so to really address this, you need these models that have this interaction effect. And economists have been kind of trying to rush to add in that interaction. 

But it’s been hard for us to do this in a credible way in real time. And so there are then lots of epidemiological models, but arguably, some of these economic trade offs that are so important for policymaking weren’t really integrated into those models in a systematic way. So hopefully, we can learn from this and provide even better models for the next pandemics, which we will no doubt face. 

CONSTANTINE YANNELIS: Politicians often don’t talk directly about different models, but implicitly, they’re very important. If we think about when the executive branch makes various decisions about, for example, how to react to a new pandemic like COVID-19, there’s typically some model in the background of various economic damages. So even if they’re not what people see on the nightly news every night, models are in the background and they’re tremendously important. So how we think about different models when they often conflict with each other is a tremendously important and often neglected part of policymaking. 

LARS PETER HANSEN: Right. Target for a lot of this stuff is not necessarily the frontline. Policymakers are all talking to the people, but it’s their top advisors behind the scenes trying to help them frame the coherent policies. And I think our best shot is trying to influence those people. 

EDUARDO PORTER: And I have this feeling that the output of your work– kind of like you’ll have to meld it into kind of a political economy setting, because the actors that are going to actually be implementing these decisions live in political systems, right? I’m thinking right now, the decision to open or not schools is probably going to be very affected by what parents want and by what teachers want. And they have different sets, I think, of fears and objectives. 

I wonder– decision theory kind of like needs a side of political economy, political science, right, in order to be effective as a tool for policy. 

CONSTANTINE YANNELIS: Yes and no. I think some of the factors you bring up– for example, different groups desiring different things– can be modeled in the objective function, right? Policymakers might put more weight on one group or another. For example, if parents want or don’t want schools to reopen, more than single individuals who can weigh their objectives differently. 

I think the political economy concerns come in more if we’re thinking about the benefits of incorporating model uncertainty to decision making to make sure that policymakers don’t just select whatever models supports the conclusion that they want for personal or other reasons. 

EDUARDO PORTER: And listen, Lars, in a paper that you co-authored, I found this very interesting point that using decision making rules not only helps make better decisions, but it also might help explain the decisions to the public. And that seems to me very valuable, given the kind of entrenched mistrust in government that one sees out there. In the context of COVID, how might this approach be used to like build public buy-in for things like closing businesses or mandating the use of masks and stuff like that? 

LARS PETER HANSEN: Yeah. The past has been policymakers have really shied away from confronting uncertainty in terms of predictions and forecasts and the like, because they thought they had to communicate their policies to a public. And they thought the public would prefer people that sounded like they knew what they were doing. And so therefore, there’s this bias towards policy advisors giving advice with incredible confidence, because that’s what politicians wanted to hear, because that’s what politicians could then sell to the public. 

Unfortunately, that leads to some perverse outcomes. And experiences like the financial crisis, experiences like COVID, I think, helps the public to understand that we’re dealing with phenomenon that if someone’s going to tell you they understand everything, you shouldn’t trust them. And that uncertainty is there. It’s there, we have to live with it, we have to deal with it. 

And so rather than hiding the uncertainty, events like this help us to put it out there in the open. And that alone can really help the decision making process. And if the public is willing to accept the fact there’s serious uncertainty there, they can also then start understanding and embracing and accepting policies that they’re not inside grounded in some pretense of false knowledge, but rather, actually, ones that are kind of reasoned through based on different possible outcomes and scenarios that are within the realm of the reasonable predictions for models. 

EDUARDO PORTER: When you look out at the horizon of COVID, of the panorama of COVID, and of the different policy responses because it’s in all over the map, where are the no-no’s? Where do you see the flaws in the policy response? 

LARS PETER HANSEN: I mean, the biggest no-no’s I’ve seen are the policy response where policy makers grab on with great confidence to optimistic forecasts and just go with it until they’re blatantly proven wrong with a high cost to society. 

CONSTANTINE YANNELIS: I think that a fair number of people now take a given model and try to put on certainty bands about their predictions of it, meaning that less that’s done is looking across models and trying to do that in much more systematic ways, which I think for some phenomenons, is absolutely important. We do think that the financial crisis exposed limits to our knowledge of macro economics. There’s always macro economists who would tell you they predicted the financial crisis, but most of them didn’t predict the timing or the magnitude of it. So they’re kind of a superficial success stories. 

With this COVID as well, uncertainty is there. And I think the public now understands it’s sitting there. So hopefully, that really does put how do we make intelligent decisions under uncertainty more central to the radar screen of policymaking. 

TESS VIGELAND: Now Eduardo, I know that there is a lot of literature these days for evidence-informed frameworks, for policymaking using the best tools possible, using data to make decisions at all levels of government– federal, state, and local. 

EDUARDO PORTER: Yeah, the challenge today is that the data that we have at hand doesn’t really add up to a complete coherent picture that could lead policymaking in a particular direction. 

TESS VIGELAND: Right. And as we were saying at the top of the show, the pandemic is really, a perfect example of a situation where we had all this massive uncertainty at a time when huge decisions had to be made very quickly. 

EDUARDO PORTER: We have to keep in mind that economists build models to describe their real world, but these models are just that– they are not perfect representations of the world. They rely on assumptions about how people behave. They are a valuable tool, for sure. But they are certainly not a crystal ball. 

Pandemic economics is produced by the University of Chicago’s Becker Friedman Institute of Economics. Our producers are Devin Robbins and Dana Bialik. Our executive producer is Ellen Horne. Production and original music by Story Mechanics. Pandemic economics is part of the University of Chicago podcast network. I’m Eduardo Porter. 

TESS VIGELAND: And I’m Tess Vigeland. Thanks for listening.