The Feedback Intervention

The Feedback Intervention

60 people attended. 45 filled out the survey. 75% said it was ‘useful’ or ‘very useful.’ 80% reported having ‘a lot’ of fun.

The funders were pleased, but I was not. Who were the 25% of people who did not find the intervention useful? I was 14 years-old and accounting for my very first $5K in public sector grant money.

Evaluation was an accountability tool, not a feedback tool. It came at the end of the implementation process. I had to count my outputs (# of participants, $ spent). I had to verify reaching my self-set goals (educating adults to involve young people). Funders were the primary users of this information. Not me. And certainly not the 60 teachers, police officers, and community workers who were the participants.

What would have happened if the evaluation has also been a feedback tool? If it was the intervention in and of itself?

That’s one of the big ideas behind some of the more widespread social programs – like Scouts, Alcoholics Anonymous, even Weight Watchers.

As a Scout, you sign-up to a very explicit set of values and goals. Abstract values like ‘good citizenship’ ‘leadership’ and ‘respect’ take form in goals like “performing CPR” and “helping a stranger.” Peers and scout leaders help you to close the gap between your goals and your behaviors. They set the norms; model the standards; offer guidance. Badges denote goal attainment. Evaluation isn’t some discrete event at the end of a financial year. It’s not simply a survey that you never see the results to. Evaluation is inbuilt into the activities themselves – and the data serves both to enable change, and to validate a program logic.

As a member of Alcoholics Anonymous, you also sign-up to a pretty explicit set of values and goals. Peers and sponsors help you to close the gap between your goals and your behaviors. They set the norm; models the standards; offer guidance; and create a space for self-evaluation. Again, evaluation isn’t some add-on activity. It’s core to the intervention. Measurement occurs at every meeting as members stand up and share their days of sobriety, their successes, their missteps. It’s a shared, reflective experience.

What would schools, welfare agencies, family services, and public health programs look like that used feedback to enable change, rather than evaluation to account for performance?

They wouldn’t look like Haley’s school.

Haley was 16 years-old when I met her on a bus in Brighton. She was fed up. Fed up with boring teachers ignoring her; fed up with her so-called friends talking behind her back; fed up with caring for her little sister. One day it all got too much. So she yanked the red handle that said FIRE. Within hours, she was expelled from school. Hannah knew she wasn’t supposed to pull the handle. She even knew the consequences of her actions. But with one pull, she got the immediate feedback that was absent from her day-to-day life.

Schools are one of the few public institutions with a remit for learning, and yet “incidence of feedback in a classroom is very low, at best measurable only in seconds per day (Voerman et al, 2012, p.1).” And not all incidences of feedback are the same. Researchers Avraham Kluger and Angelo DeNisi (1998) write that, “The data suggests that at least under certain circumstances, Feedback Interventions can impair performance…Feedback Interventions can be double-edged swords (p.67).”

Were feedback a behavior change tool – not only a performance tool – we’d prototype how feedback works, in different settings, with different people. Indeed, we’d design interactions for giving and getting feedback that take into account contexts, goals, and tasks – all the factors that the research says matter.

How feedback works

There’s a few different theories about how feedback interventions change our behavior. One theory focusses on discrepancies: it says we’re motivated to act when we discover gaps between where we are and where we want to be. Another theory focusses on our locus of attention: it says that feedback either directs attention to ourselves, to the tasks we’re trying to do, or to the details within the task. Feedback that channels too much of our attention inward can miss the mark. That’s because we expend too much brain power reconciling the ‘threat’ and inadvertently take our eyes off the goal at hand.

Why goals and tasks matter

There is not just one kind of goal. We have mastery goals to execute a skill or complete a task. We have relationship goals to make and maintain social connections. We have goals about our  ideal selves (what we want to be). And we have goals about our ought-to-selves (what we believe we should be). Research shows that positive feedback makes us less likely to achieve goals about our ought-to-selves, whilst negative feedback prods us on. We might also frame our goals differently. We can represent a goal in terms of ‘committing’ to a desirable state (say, losing weight). But we can also represent a goal in terms of ‘progressing’ towards that desirable state (say, eating less this week). Studies suggest that positive feedback increases our motivation to act when we’re in commitment mode (we’re encouraged to keep on keeping on), but actually decreases our motivation to act when we’re in progress mode (we think we’ve done enough).

Alcoholics Anonymous is all about commitment goals – as a member you are signing-up to a sober lifestyle. Positive feedback about your own and others’ past successes supposedly encourages your sobriety today. Scouts uses positive feedback in a similar way – to encourage leading a life aligned to the values. Weight Watchers, on the other hand, is all about progress goals – as a client you get points for exercising and points taken away for eating too much. Negative feedback on your food intake supposedly increases your motivation to act (Fischbach, Eyal and Finkelstein, 2010, p.519).

Why emotional responses matter

There’s one big, unknown variable when it comes to giving and getting feedback: us. Each of us have different mood states, trigger points, and past experiences which shape our emotional response to feedback. Shame, guilt, embarrassment, and pride are what are known as the self-conscious emotions – and they are generated when we self-reflect or self-evaluate. “They function as an emotional moral barometer, providing immediate and salient feedback on our social and moral acceptability (Tangney, Steuwig, and Mashek, 2011, p.2). But not all of these emotions motivate us to constructively act. For example, shame is linked with low-self esteem, depression, anxiety, suicidal ideation, and a whole host of other mental illnesses.

Too many of our public systems and welfare programs are predicated on shame and embarrassment. Take Haley’s expulsion. School administrators hoped to send her a message by inducing embarrassment and shame. They thought: if she knew how ‘bad’ it felt to ‘let people down’ she wouldn’t do it again. Only school administrators failed to take into account Haley’s context. She felt shame and embarrassment just being at school – ignored by teachers, bullied by peers. For her, expulsion was relief. And she was likely to do it again, and again. So many of the young mums and families I’ve spent time with over the past 5 years continually repeated their self-destructive behaviors – perhaps in part, because they kept getting the same responses. The only time they were explicitly acknowledged was when they did something ‘wrong’ – when their kids acted up, when child protection came to visit them – but they were hungry for interaction, for attention, for some sort of validation.

Segmentation needed  

Haley’s school never learned that expulsion was the wrong kind of feedback to positively shape her behavior. Nor did the welfare programs and family crisis services learn that threats often negatively shaped behaviors. They gave the same punitive feedback to everyone they encountered. (That, after all, was ‘fair’). I too have given the same critical feedback to all staff members – without always taking into account how they framed their goals and their tasks, or how they emotionally responded. What I was lacking – and what Haley’s school was lacking – was a good segmentation. A way to understand for whom our feedback interventions did not work, and some ideas about why.

To get to that understanding, we need an entirely different kind of feedback intervention. A kind that’s not just directed downward – at the behavior of end users, be they students, families, staff members. But a kind that looks at a system’s collective behavior – at the behavior of students, teachers, principals, and parents; at the behavior of families, social workers, managers, judges; and at the behavior of staff members, bosses, board members, and external stakeholders. Haley’s expulsion was aimed squarely at her – it didn’t serve as feedback to her teachers, her peers, or her school. How, then, can we enable multiple feedback loops to intertwine?

Over the past week, I’ve watched 2 feedback loops intertwine. Those of staff and bosses. At Kennisland (the think-do tank where I am a visiting scholar), all of the staff had their performance reviews. Unlike lots of other ‘socially innovative’ organizations I’ve been a part of, Kennisland actually invests time in giving staff regular feedback and invests resources into their personal development. Staff really value these two-way conversations. Yet the conversations have their limitations: (1) the same feedback mechanisms are used for everyone and (2) individuals get feedback in isolation of the bigger system they are part of. So a staff member’s performance is gauged by their own evaluation and that of their bosses – but it is not linked to their peers, or to their project partners, or to the people they are ultimately trying to affect – teachers, civil servants, community members.

A different kind of feedback intervention might regularly bring parts of these systems together to map their concrete interactions and products – and use that to understand the goals people have, the standards they are operating under, and the gap with their actual behaviors.

As a first step to designing this different kind of intervention, I’m experimenting with a Feedback Diary. I’ll be asking all sorts of folks (staff, students, teachers, doctors, etc.) to record the feedback they give and get in a week, their emotional response to that feedback, and the goals & standards to which that piece of feedback relates. I tested the journal out last week – and was struck by how often I didn’t know what other people’s goals or standards were and how much I operated in the dark, without any explicit feedback. I’ll used the Feedback Diaries to generate some concrete ideas for segmenting feedback, and connecting feedback loops.

I’ll know we’ve made some progress when evaluation is synonymous with feedback, and performance with iteration. When I’m not left wondering why 25% of people didn’t get anything useful from an intervention – and most importantly, when I haven’t missed an opportunity to use that information to shape individual and system behavior.

In this piece, I cited…

Lisa Boerman, Paulien Meijer, Fred Korthagen, Robert Jan Simmons. 2012. “Types and frequencies of feedback interventions in classroom interaction in secondary education.” Teaching and Teacher Education, (2012), pp. 1-9.

Ayelet Fischback, Tayl Eyal, and Stacy Finkelstein. 2010. “How positive and negative feedback motivate goal pursuit.” Social and Personality Psychology Compass. (4)8, pp.517-530.

Avraham Kluger and Angelo DeNisi. 1998. “Feedback interventions: Toward the understanding of a double-edged sword.” Current directions in Psychological Science. (7)3, pp. 67-72.

June Price Tangney, Jeff Stuewig and Debra Mashek. 2007. “Moral emotions and moral behavior.” Annual Review of Psychology. (58), pp.345-372.

***

These (rather long) blog posts – what I’m calling writing pieces – are the start of a book project called The good, the bad, and the Feedback….  The book seeks to answer the questions: What is good? How do we know? All feedback welcome!