Triple feedback loop

In onderstaande blogpost onderzoekt Sarah de zin en onzin van feedbackloops, dubbele feedbackloops en driedubbele feedbackloops.

Honk!

Ding!

“No, just 20 cents more”

“Nice hat”

“Wrong direction”

It’s 9:34am, and in the 32 minutes since I’ve left the warmth of my apartment for the blustery Amsterdam streets, I’ve received 5 pieces of feedback. And that’s just the explicit kind.

“Feedback is such an all-pervasive and fundamental aspect of behavior that it is as invisible as the air that we breathe,” observes William Powers, who has written a lot about the negative feedback loop.

This is what the basic negative feedback loop looks like.

We are constantly collecting information about the state of the world. How people look at us. What street signs say. The numbers on the cash register. We compare this information to our various goals. Not looking like a tourist. Getting to work in a new city. Buying a coffee. We perceive discrepancies between how things are going (our actual state) and how we’d like things to be going (our desired state). And then we act in ways that we believe will cause the real world to move closer to our desired state. In other words, we change our behavior. To look like a local, I cross the street with everyone else. I do not wait for the little green man to shepherd me.

The basic negative feedback loop is behind a growing gaggle of behavior change tools – from the Nike + iPod sports kit that measures the distance and pace of your running; the Weight Watcher Tracker that records your food intake and activity output; the energy management apps that capture your home power usage. Just last week, Bill Gates made the case for a national teacher feedback system. He cited the results of a small-scale pilot, the Measures for Effective Teachers, which draws on test scores, student surveys, and peer observation to give teachers information on their own performance.

The assumption is that information is the missing link between our desired and actual states. Technology can help us to close the gap – both by collecting relevant data, and turning that data into actionable information. But what happens when that information leads us to move even farther away from our desired state? Or when our desired state isn’t actually so desirable?

Jess is a 20-year old mum living in a nondescript South Australian suburb. A product of the foster care system, she knew what it was like to grow up in a house that was never a home. She wanted something different for her son. That got expressed as protecting him from the system that raised her. When social workers came knocking on her door, she refused to engage. They sent threatening letters. The information in that letter confirmed her worst fears. So she ran. Her behavior became information that only confirmed the social workers’ worst fears. Jess’ desired state was emotional safety for her and her son. The social workers’ desired state was physical safety for her son. Neither desired state was complete. And information only fed their existing conceptions of safety.

“Information feedback about the real world is not the only input to our decisions,” academic John Sterman reminds us in his review article “Learning in and about complex systems.”

That’s where the double feedback loop comes into play.

The double feedback loop recognizes that we interpret and act on information in a context of existing rules, cultures, and institutions. Such structures reflect how we think the world works. In other words, they are physical manifestations of our mental models. Jess’ social workers interpreted and acted on information within a beleaguered institution constantly reacting to bad news events. The rules and processes reflected a mental model that was all about short-term harm minimization – for the state. So how do we make our mental models visible, and contestable? And how can we use information to challenge how we think the world works, rather than confirm our existing beliefs and biases?

That’s what design thinking is supposedly all about. Using different kinds of data (ethnography not just statistics). Visualising that data (drawings & photos not just text & spreadsheets). Eliciting empathy. Prototyping new responses.

But what happens when our mental models are more virulent than our methods? How do we move beyond the superficiality of one-day ‘insight’ workshops and multi-colored post-it notes to deeper values-based work?

Rochelle is a 20-year veteran of a large government department. She worked her way up from a junior analyst to a division manager, in charge of a sizable staff and a chunky budget. Her boss is a 25-year veteran of the same department. They were charged with making services more customer friendly. Their customers happened to be nearly every Australian: any one who interfaced with the health or benefits system. Over the course of a 9-month project, I got to know some of their customers. The Burundian family struggling to learn English and find meaningful work. The Kiwi family living in a 3-room trailer. The single mum wanting to return to school. These stories – this information – confirmed what I already thought: the welfare system was inadequate. Cash payments alone could neither build capability, nor leverage the system’s most powerful resource – its users. For Rochelle and her boss, these stories – this information – confirmed what they already thought: the welfare system was inefficient. Far from expanding its functions, we needed to clamp down on the manipulators and system abusers – the Kiwi family who was actually ineligible for support, the Burundian family who wasn’t looking hard enough for work.

All our information – ethnographic write-ups, photographs, journey maps, and snippets of film – did was further entrench pre-existing viewpoints. Because this information was so at odds with all the other information flows within the system, it could be discarded as an aberration. So statistically insignificant that it couldn’t possibly disrupt such a big system.

What Rochelle and her boss helped me to see (a year after the fact) was that we needed to design more robust feedback loops. Feedback loops that could not only help expose how we each thought the world worked, but could reveal how we each wanted the world to work. Feedback loops that could challenge our values & ideals, not just our practices.

This is what I’m calling the triple feedback loop.

Of course drawing a loop on paper is much easier than bringing it to life.

Sterman suggests that ‘virtual worlds’ are good settings for activating feedback loops. He talks about ‘virtual worlds’ as places where “decision makers can refresh decision-making skills, conduct experiments, and play. They can be physical models, role plays, or computer simulations…The virtual world allows time and space to be compressed or dilated (p 317).”

I met Jess, Rochelle, and her boss through such a virtual world. These virtual worlds go by various names: social design studios, social innovation centers, solutions labs. Working In, With, and For the Australian Centre for Social Innovation (TACSI), we co-designed, role played, and ran live simulations (i.e prototypes) of solutions. But we rarely (if ever?) shifted peoples’ mental models. Nor were we always explicit about our own mental models. Instead we worked with the willing, with stakeholders who shared our mental models – people mostly outside of systems, or marginalized within their systems. And when we’ve worked with stakeholders who didn’t share our mental models, it’s been exhausting and too often fruitless.

What, then, can we do?

Here again, Sterman gives us a few tangible hints. He names 9 barriers to setting-up meaningful feedback loops. We never designed TACSI to systematically address these barriers: Externally, be it with families, older people, aged care organizations, or service providers; Internally, as a group of designers, social scientists, community developers, business analysts, project managers, and a chief executive. If anything, the TACSI experience taught me that unless we can model triple feedback loops from the inside, we can’t expect other systems to do the same.

Modeling triple feedback loops will take overcoming…

Dynamic complexity

There are loads of feedback loops operating within any one system (be it a workplace, a family, a government agency). Think of how many people there are, the different roles and interests they represent, and the number of information sources. To further complicate matters, there are significant time delays between when we take a decision and when we see its effects. Some actions yield irreversible consequences. And it’s hard to hold variables constant. Were we to design around this barrier, we’d need to think about how to make existing feedback loops visible; the timing of simulations (in prototypes to date, we’ve set an arbitrary 3-month time frame); and how to run prototypes as full-fledged experiments.

Misperceptions of feedback

We’re human, and our brains can only handle so much information at one time. We have a hard time making inferences, and drawing nuanced conclusions. Were we to design around this barrier, we’d need to think about how to sequence and chunk information differently. We might take a more layered approach, over time, rather than providing an onslaught of new information at once. And we might also build people’s capacity to understand how feedback & the brain actually works.

Flawed mental models

We’re not very good at detecting the relationships between different bits of feedback, or making sense of correlations. To make matters worse, we tend to attribute people’s behavior to their personality rather than to situational factors. This is known as the fundamental attribution error. It’s problematic because we end up treating individuals as the problem, rather than re-designing the environments that surround people. Whilst we often looked to environments to explain end users’ behaviors, we rarely did the same for stakeholders, colleagues, or project leaders. Were we to design around this barrier, we might apply the same ethnographic methods used with end users to organizational users. This suggests an alternative way to structure teams and projects.

Defensive routines & interpersonal impediments

Giving and getting feedback isn’t easy. It can clash with the views we hold about ourselves, or what we deem to be culturally appropriate. Often, our responses to feedback are unconscious. We fold our arms, we roll our eyes, we change the tone of of our voice. In such settings, it’s all too easy to hold back, to avoid the discomfort, and to give in to a sort of groupthink. Were we to design around this barrier, we might develop a range of routines and communication patterns within teams and stakeholder groups. We might test how people respond best to feedback. We might find ways to hold up a mirror to ourselves. We might make more room for the emotional reactions – rather than expect the rational.

**

I’m now on the hunt for programs, services, and organizations that have designed feedback loops around some of these barriers. I’ll spend time understanding how they work, when, and why. If you’ve got any leads for me, please get in touch: hello@sarahschulman.com

These (rather long) blog posts are the start of a book project called The good, the bad, and the Feedback…. The book seeks to answer the questions: What is good? How do we know? All feedback welcome!

Sarah Schulman

(this article was originally posted here: sarahschulman.com/triple-loop-feedback)