Organizing Dialogue, Experience and Knowledge for Complex Problem-Solving

Weather and Uncertainty: Warn or Wait?

September 5th, 2011

Crossing the Rubicon?

Crossing the Rubicon?

One of the striking things that I learned about Americans when I began doctoral studies in the field of Communication is that there is a positive identity function to talking about the weather. If you’ve got to interact with a stranger one thing we all experience is the weather. Rather than being superficial, talking about the weather is a small ritual of interpersonal communication done in similar ways by so many different kinds of people that it aggregates into the significantly large effect of contributing to a common sense of shared national citizenship.

Source: (retrieved 4 September 2011)

Source: NOAA (retrieved 4 September 2011)

The economic cost of big weather events is increasing. What about the social costs? What I mean by “social” are the quality and types of relationships among individuals in the United States, between these individuals and the publics we make up and belong to, the scientists and businesspeople involved in the weather enterprise, and among all these groups and the government. Whether or not the degree of public awareness and engagement with the weather is indicative of climate change or is merely a statistical blip that will wash out over time, media hype and active debate suggest a ripe opportunity for intervention in improving the emergency infrastructure so that everyone can better prepare and respond more resiliently to severe weather events (and other disasters).

Problem Definition: A Matter of History

Anyone who’s done research knows that how you ask the question has a lot to do with the results. The challenge of intervention is to ask the right questions; to ask these questions at the right times and in the right places, among the right people, and about the right thing/s; and then to see the discussion through. In Thinking in Time: Uses of History for Decision-Makers, Neustadt and May explain and illustrate the necessity for conceptualizing in terms of timestreams.

Thinking of time in such a way appears from our examples to have three components. One is recognition that the future has no place to come from but the past, hence the past has predictive value. Another element is recognition that what matters for the future in the present is departures from the past, alterations, changes, which prospectively or actually divert familiar flows from accustomed channels, thus affecting that predictive value and much else besides. A third component is continuous comparison, an almost constant oscillation from present to future to past and back, heedful of prospective change, concerned to expedite the limit: guide, counter or accept it, as the fruits of such comparison suggest. (1986:251)

I summarize Neustadt and May’s mini-methods, and argue for their application as a strategy for design, in this blog entry: Implementing an Organizational Vision through Thinking in Time.

Problematic Moments Signal the Potential for Powerful Change

WAS*IS problem definition exercise, 10 August 2011, at UCAR in Boulder

WAS*IS problem definition exercise, 10 August 2011, at UCAR in Boulder

The most lively discussion held at the 2011 WAS*IS Weather and Society Summer Workshop involved how to improve tornado warnings. Even though I was not formally in my action researcher role, I had been authorized to do ‘live blogging,’ so I was taking copious notes and paying close attention to the discourse and dynamics as they were unfolding. After this activity on August 10th (the 6th day of the workshop), Bob described it as “a barnstormer of a discussion” and Justin noticed a shift: “It’s all been happy-go-lucky so far, [and now] we’re going to have some disagreements – but the good thing is we all respect each other.”

watch your headFrom my vantage point, we had just gone through a group-level problematic moment. I’ve lived through several of them, and so have you – any time a whole group suddenly falls silent, or most members of the group spontaneously burst into talk – some underlying issue that effects everyone has somehow been tapped. James Cumming suggests these are “the same kind of moment that Bergson calls “durée:” a moment of silence that prepares the way for discourse, possibly new discourse, and with that the possibility of change.” [Noise is another kind of silencing: din obliterates sensible sound.]

Usually problematic moments are passed over, politely or awkwardly ignored. In our case, the discussion had already gone longer than scheduled and intruded into the break time. The moment of simultaneous talk became the excuse to end the activity and move into break and on with the rest of the program. Exceptionally, however, the vast majority of WAS*IS workshop participants kept talking with each other: over the next few minutes I counted at least thirteen animated interactions, from pairs to trios and one group of four, whose conversations continued as if there had been no interruption. If only we could have captured each of those unique conversations!

tornado shelterBarnburning: The “Warn on Forecast” Concept for Tornadoes

I typed as quickly as I could. This section is mainly the description provided by Workshop Leaders for a heuristic activity regarding problem definition. We were not supposed to try and solve a particular problem, rather, we were charged with the task of applying our collective intelligence to as many components of the issue as we could imagine to question.

a) The current system accounts for detecting a tornado threat (and issuing a warning) 0~45 minutes before it hits.
b) 10+ year goal is to increase this to ~2+ hours . . .
c) How????? Through better models, etc…. [although] “we have models that are coming out of our ears”


  • Kenny clarifies: a specific storm that does not exist yet, a particular threat in a given location…
  • Dan N: an area, maybe Boulder …
  • The default WarnGen shape – “I’m not crazy about that shape, but that’s sortof what they look like…”
  • Current system:
    • Assumes the public is homogeneous
    • ‘one size fits all’ – “you’re either in the warning or not, at risk or not
    • Purely meteorological polygon:
      • Purely bimodal
      • YES you are in or NO you are not
      • Orange dot could be (a popular state park, boy scout camp, county fair, mobile home park, school, hospital)
      • “ugly tornado, or we think it might be”
      • This is about the communication of uncertainty
      • Jay: “you might want to show where the expected tornado location is”
      • Bob: “I have a problem with that orange dot, apparently I have less value (having a bbq with my family) than the Boy Scout Camp.”
      • Mark: “How many dollars it costs for every hour somebody is under a tornado warning” ~ there is “an economic cost involved.” “It’s not a no-brainer.”
      • Jamie: “I’m being undervalued because I’m not in the polygon, but putting you in the polygon means increasing the area of uncertainty” …. five times you’re in the polygon, nothing happens, the 6th time you decide to ignore the warning…
      • Assign percentages?
        • Blue Dot: State Park
        • Green Triangle: Mobile Home Park
        • Red Star: State Fair Grounds
        • Green Square: College Football Game
        • “What is your probability threshold?” “How sure are you that locations inside the box will receive severe weather?” At receding distances beyond the boundaries of the polygon…
        • Greg: specific only to this one storm, not taking into account future/other storms
        • Dan N: only one/current to keep things simple
        • Talia: “How fast might a storm like this move,” influencing when you would move the boundaries…
        • Dan N: 20-30 mph (roughly)
        • Kenny: maybe there’s an on-the-ground report
        • Robert: that measure of speed is an average, some can move up to 70 mph
        • Dan N: just a simplified model for the purposes of the exercise: “Storms can do all kinds of crazy things.”
        • Ben: “Is that more of a fear that meteorologists will miss an event that impacts a lot of people, or is it to make the overall system more dynamic?”
        • Dan N: “Hold that question!”
        • Dan N: add  intensity of tight game – Buffaloes vs Huskies
        • Matt: are they playing in Lincoln?

Probabilistic Hazard Information (PHI)

At this point, it seems most of us have grasped the instructions and the scope of the example.  Now the types of question begin to shift, becoming more diagnostic: the group begins to address the task and interrogate the scenario.

  • Uncertainties/probabilities addedtornado outbreak
  • Longer lead times
    • Warnings for lesser certainty can be issued
    • 1-by-1 pixels

Holly: What happens if the storm regenerates? Could be misleading to the people in the blue?

Dan N: It’s extremely more complicated than this; storms could be popping up all over the place….”maybe in your discussions, you could kick all these ideas around” . . . presented this way “for simplicity’s sake” . . . aware “maybe that’s the problem”

Spinney: “Is each a different product?”

Eve: We had an advanced WAS*IS  about this in 2008.  “I thought, people just need to know each other, as soon as this gets explained, social scientists, emergency managers, hospital administers… total flashback, and a little of post-traumatic stress….. an anthropologist and others asked, “Why didn’t you ask us what we might need?” “why did you assume that this is what we wanted?” . . . “No one has really recovered from this yet….” Our assumption that …. How much trust in the NWS …. It would be magic.  “This is really hard; if we could make progress on this, it could really change the way these things go: we’ve got private sector people in the room, more social scientists in the room, much more sophisticated understanding… the dream was, we’ll explain it to you and you’ll make it easier for us….”  Look at webpage for 2008, _______ (?)…. She doesn’t do this anymore, she went back to lightning. This was too hard.”

Susanna: one of the potential mistakes here, going from warm colors to cool colors, these things we’re used to from – intensity indicated by hot colors…. If blue I’m thinking the weather is going to be mild – the color scheme of a storm

Dan N: “light rain….moderate…. etc:

Robert: the color scheme is resonant of what the Dept of Homeland Security used… we were always on green or blue?

Kenny: rules of cartography were not considered when these codes were put together – they spend a lot of time learning about design: hues, color, intensity, perceived meanings… “a mismatch between the product and what’s intended”

Dan N: “be careful quoting me on representing a weather forecast” because that’s not the point here, which is to get us into an exercise

  • 1 km by 1 km grid boxes,
  • this product will give me a percentage, e.g., 17% of being hit by a tornado,
  • two primary issues –
    • 1.  much longer lead time of 2-3 hours, but then we have
    • 2.  increased the probabilistic warning information

Bob: I’ll start looking for secondary info, the longer lead time will reduce the urgency of threat . . . weaken the intent of warning

TASK PROMPT: Do we Warn on Forecast?

ZillianalienThe group is immersed in imagining variables, conditions, assumptions, the breadth of brainstorming is phenomenal. Some members of the group begin to question why, if, when, how, and who should be warned – or not.

  1. What problems were defined?
  2. Does this concept appear to solve the defined problems?

Susanna: This is focused on problem at the local level; but what are the factors at the higher level? Do we have to stay in this paradigm, what do to make it better? How introduce new factors at the higher level – because this isn’t even the paradigm we want to be working within?

  • Who needs more lead time? Football stadium… it takes a long time to evacuate…
  • Gaby: clarification ~ issues addressed in the powerpoint or the exercise?
  • Rebecca: both
  • Bev: we’re assuming lead time will make a difference? Will lead time move into actions that people can take to protect themselves?
  • Gaby: who is this for? May benefit some individuals more than others ~ meteorologists, or comm between emergency managers, but not the public
  • Dan N: generic answer is for everyone – currently tornado warnings are for everyone.
  • Rebecca: audience matters
  • Gaby: if we’re thinking public there are things to consider, if emergency mgmt. personnel there are others to consider
  • Chad: a convective outlook (meteorological) – the meat of the argument…. But jargon-laced to the public. More detail oriented for those people and more general for the public, some nerds who will love looking at the convective outlooks ~ maybe that’s an analogy?
  • Dix: a lot of those products were not made for the public, but they are out there…. We still need them, do you use them/convert them for use by emergency managers….
  • Chad: Interpret and spin into message for their audiences?
  • Dan N: from a physical science perspective, this all makes great sense
  • Greg: will the idea of uncertainty and probability be understood? In the location? In the timing? Where is the uncertainty? Will it appear?
  • Susanna: would it be helpful to communicate the numerical probability?
  • Rebecca: meteorologists assume this will happen, if communicate the numbers it will work
  • Susanna: doesn’t work
  • Greg: that’s the assumption
  • Jay: “whenever we do something like this in real time, we disregard the bigger threat on the north side of that storm which is the hail…. Telling people to get indoors but we should tell ‘em to get their cars indoors to protect from the baseball sized hail”
  • Talia posed something
  • Dan N: why issue tornado warnings in the first place
  • Jamie: give people the info and let people make the decision
  • Rebecca: empower decision-making as the goal or “I know what the right thing is, you do it?”
  • SJK: should be recording this conversation
  • Susanna: bureaucracy
  • Brittany: “will see what I can do”
  • Ben: who decides? Authorities or whose responsible for their family – we’re experts, analogy with a doctor, you need this surgery or here’s the options with their probabilities, people want advice that’s what a tornado warning does…
  • Dan N:
  • Susanna: where we’re miscommunicating is audience ~ doctor, patient, FEMA, hospital administrator, we keep forgetting which public when giving examples, the assumption keeps assuming there is one audience, how do we adapt products to multiple audiences rather than just one
  • Dan N: if we go this route, what Eve was talking about… if we go this route, how (and what effects)
  • Susanna: You’ve shot yourself in the foot, “this is for everyone in the US”
  • Dan N: defends
  • Susanna: must have different entry points in web design, cannot start
  • Greg: people have scales too, just like weather does
  • Justin: what is a problem, what do we do with: bimodal you get hit or not by a tornado same with rain… must have confidence levels – problems with the current model; nothing else in weather is done bimodally
  • Robert: Check out the Warn on Forecast webpage
  • Dan N: let’s not get stuck there
  • Rebecca: intriguing to analyze their assumptions
  • Brittany: assuming that the public knows what their context is – people know if they’re in a football stadium
  • Dan N: you’re bringing it to another level, how could it be done better (isn’t this what we’re supposed to be doing?)
  • Bob: are they just doing this for themselves to show people how smart they are? They’re missing it now, more of the same will help? Not.
  • Jamie: both, ½ the org has one motivation and ½ another (to help the public or show off)
  • Alan: if people aren’t doing something at 100% why are we thinking they’ll do something at 17%?
  • Bev: belief that more/better will be sufficient. What do we know that will empower people? A personal relationship? Something else?
  • Dan N: “We can definitely kick around a lot of problems, but does it solve it?”
  • NO.
  • Kenny: “bring out some of the inherent tensions: In science, if someone asks me to define a problem, what are some…. E.g., “Do people have enough lead time?” there are assumptions in there…. Are these even valid questions to be asking?”  What the weather service would like, and what problem needs to be solved?  “Is this even a problem?” We need a statistical technique to show this before we go to the second stage…. Academics are asking, ‘what the hell are we doing?’ Need to apply a null hypothesis test to, before we even proceed.
  • Rebecca: often we have the data that shows how people do/don’t respond… we have contra-evidence
  • Kenny: assumption that the public needs tornado warnings – I’m not sure this has even been demonstrated. Juicy meta-problems… as we’re attempting to uncover the comments
  • Rebecca back to Jay: what are the threats we’re supposed to be addressing?
  • Dan N: the physical scientists bring up the natural phenomena (hail, etc), social scientists – do people even understand these things?
  • Dan N: “Well, it’s 10:00.”
  • Rebecca: validity – some scientific accuracy or truth, the probabilities are not well-calculated…. Not scientifically-sensible.  “You can’t assume the science is perfect, you make a lot of assumptions with science.”
  • Dan N: current RQ ~“How to draw this box?”
  • Matt: “I’m sitting here as a broadcaster feeling job security. You cannot make a policy that is going to fit every situation. There is always going to be humans involved, no matter how good computers and modeling gets – you can’t cover every situation that comes up…”
  • Rebecca: what is the larger problem definition?
  • Dan N: “This is a real one. This is real life.”
  • Jamie: “When it comes to high profile warnings, that really is the identity of the agency. A lot of people are going to come to the table with tremendous emotional attachment or baggage, to them, this is who they are. The tornado or hurricane warning is how they define themselves.”
  • Dan N; “Meteorologists are almost born thinking, ‘It would be great if we can give 5 hours lead time…’”
  • Greg: how organic systems interact: atmosphere, oceans, cryosphere, land – a classic graphic. The human aspect needs to be put into that five-part connection; the problem is that we’re dealing in two different systems that are always evolving, we always have to make assumptions to simplify…
  • Jelmer: “everybody needs coffee, I know” “ We’re discussing the things, but they are obviously not isolated to this room, there are people doing research about it, already, I hope… a list of references? Otherwise we are talking about it now and maybe forget it.”
  • Bob: question, insane, bear with me: “Does  a reduction in lead time increase probability increase risk…. Short-term decision-making… expose people to greater risk…. Short-term lose more lives & property but over time…. “ a recency effect….
  • Steph H: larger venue decision-making for large institutions: schools etc ~ some sort of probabilities info could help, they may make better decisions than the usual public
  • Susanna disagrees: school principals? Probably not
  • Steph H: longer lead time, seems less life-threatening, not beneficial, more adverse effects (their speculations)
  • Bev: school officials may have more anxiety
  • Susanna: making a joke, can’t assume people with more education automatically understand probability, but some audiences do need more lead time. Bob – for some people it won’t make a difference but for some it will, the hospital administrator needs it
  • Brittany: for emergency mgmt a very useful tool, not the general public
  • Chris: I can see giving the info to an elite group, then feeds rumors – we’re preparing for such and so, but nobody knows, “It’s gonna get out whether we put it out or not”
  • Amy: ethical, legal
  • Chris:


road sky thunderstorm

Several talking same time: Justin-Chris, Amy-Robert, Steph H (at least)
Rebecca H – the importance of the proactive step
(Mark-Dix, Chris, Dan N, Kenny…Rebecca….)

SLOWAn ethical dilemma – withhold info?

Tell who? (and who not?)

Ownership of taxpayer to the info, responsibility to provide ~ Chris on the shock of imagining the possibility of not telling – proving Jamie’s point about identity.

Leave a Comment

Categories: PM dynamics, Reflexivity, Resiliency, Science of Team Science, Series

Leave a Reply