The nasty questions

Sarah kijkt in onderstaande blogpost terug op de tijd dat ze in Australië het Radical Redesign-team leidde en nieuwe concepten ontwikkelde zoals Family by Family en Weaver. Ze reflecteert op de feedback die ze kreeg. Het spoort haar aan om zich bij KL te storten op het project ‘The Good, the Bad and the Feedback’. Ze gaat op zoek naar manieren om ‘goed’ te bepalen. Kennislands projecten dienen daarbij als inspiratie. Houd het werk van Sarah Schulman vooral in de gaten de komende tijd op de KL-site.

The nasty questions

Are we doing good?

That’s the big question I’ve set out to answer in 2013.

It all started with a sneaking suspicion. That the value we were creating was less than the resources we were expending. That suspicion turned into full-on moral panic. Disguised as impatience and impudence.

I wanted to know whether the Radical Redesign team I co-led at The Australian Centre for Social Innovation was doing good – or at least good enough – to warrant the nearly $1 million project price-tag the team of 9, the new studio, all of it. Yes, we got plenty of positive feedback from end users. Adele, who cares for her father with Alzheimer’s, loved meeting Coralie, whose mum also had Alzheimer’s as part of our prototype called Weavers. “It’s so good chatting to someone who understands.” But is self-reported satisfaction a ‘good’ proxy for behavior change? As Timothy Wilson points out in his book, Redirect, people are pretty lousy at assessing whether things are changing for the better. And even if self-reported satisfaction turned out to be a good proxy for individual impact, was the social impact enough? Were we changing outcomes across a whole population group? Were we changing the way existing services worked? Were we changing the flow of resources?

What standards do we use to judge whether our work is good, or good enough? How do we measure good? How do we iterate when it’s not good enough?

Since starting InWithFor (a mobile social problem-solving lab) we’ve talked about ‘good work’ as solutions that: (1) enable people to thrive, (2) are bottom-up, (3) leverage peer-to-peer learning, and (4) are spreadable. But we’ve never addressed some of the conflicts between ends and means. That became particularly true over the last 15 months, as we took on our biggest project to date: improving outcomes for people in caring situations.

There was Maggie, who cared for her partner with a brain injury. She rarely left the house. She had few friendships or relationships with family. She used dark language to talk about her reality. She had “come to terms” with her lot in life, and didn’t want anything more than a clean house and good medical care. Her partner had difficulty speaking. Most of the time, he sat in his room watching TV, alone. We would not say Maggie or her partner are thriving. Theirs is not the good life. We actually have a point of view about what constitutes a good life. How do we reconcile our views with a bottom-up, co-design approach? Do we gauge ‘good’ work by meeting Maggie’s standard or by meeting our own? How can we ethically shape her preferences?

There was Joan, who cared for her husband with a stroke. Joan’s concept of thriving was traveling, spending time with her grandkids, sleeping, and eating well. To move closer to thriving, she made the tough decision to place her husband in a nursing home. Going into a nursing home thwarted her husband’s thriving. For him thriving was remaining at home, with family, retaining some independence, eating the foods he liked, having good company. Is it ‘good enough’ to improve one person’s quality of life at the expense of another? Do we work to the lowest common denominator?

There was Helen and John, a couple coping with Parkinson’s disease. Like Adele, they were paired with a volunteer Weaver who had caring experience. But their Weaver, Tricia, also had professional experience as a counsellor and life coach. Tricia helped them to set goals, to be accountable, and to troubleshoot system blockages. Whilst the peer-to-peer aspect of the program enabled rapport building, it wasn’t the component that directly led to change. It was Tricia’s professional skills and experience. How do we uphold our peer-to-peer principles, and invest in what works? Where do we draw the line between principles and pragmatism?

There was Greg, caring for his wife Gene. Money was tight. They stuck mainly to themselves. And they rarely signed-up to anything in the community. We couldn’t get them to sign-up to any of our prototypes either. The folks we did engage were, on the whole, more educated, more community-oriented, and more active service users. How do we engage the early adopters and not increase inequality? (You can read a bit more about the inequality conundrum in my last blog post).

And then there were all the existing carer support programs and services. There are hundreds of respite and carer support providers in Australia. We became yet another provider. We could position ourselves as an alternative to existing services – but lose stakeholder support, becoming a competitor rather than collaborator. We could try and shape existing services – but the incentives within the system were at odds with an outcome-oriented, bottom-up, peer-to-peer approach. We could work to dismantle the existing system – but risked doing little for Maggie, Joan, Helen and John. When do we work to enhance the current system (e.g Play Works)? Develop an alternative to it (e.g KIPP schools)? Dismantle it (e.g De-institutionalized mental health)? What’s the balance between changing outcomes and changing systems?

Social critical Ivan Illich warns us that these two goals – changing outcomes and changing systems – are incompatible. Writing about our schools, Illich argues for dismantling rather than reforming our formal institutions. “Illich makes us wonder about the value of past efforts and future plans: Am I part of the problem? Does my work obscure the real issues? Every time I actually help a school to improve on its treatment of children, do I also help to perpetuate the hidden curriculum? These are nasty questions to ask oneself,” Neil Postman, a school reformer, writes in response to Illich’s provocations.

We could all ask ourselves the same nasty questions. When does our work obscure the real issues? Are incremental efforts perpetuating systems which are fundamentally broken? What are some ways to gauge whether we are doing good? And most importantly, what are ways to act our way through these ethical dilemmas?

That’s the focus of the tentatively titled project:
The Good, the Bad, and the Feedback.

The project will look at how we determine, measure, and improve on what is ‘good’. At an individual level – in our relationships, families, workplaces. And at a systems level – in our programs, services, and institutions.

My premise is that feedback is the basis for learning and change. Unless we know how we are going, and how we could be going better, we cannot intentionally act.

My contention is that feedback is not the same as evaluation. Evaluation aims for accountability. By and large, it’s a technocratic exercise. Not an ethical one. It’s wrapped up in the evidence-based culture that uses words like effective and efficient to circumvent the deeper (more contentious) questions of what is good, fair, just, etc. Design thinking, social innovation, and social entrepreneurship do not give us a concrete ethical framework to determine what is efficient or effective, in which contexts, be it working with older people in Adelaide, social workers in Brighton, or public servants in Washington, DC.

It’s time to develop a first framework. To do that, I’m on the search for positive deviants. For individuals, families, teams, programs and institutions that have robust feedback loops – that have come up with ways to determine what is good, measure it, iterate their interventions, and reset their standards. Some will be in the social sector. Others will not. I’ll seek inspiration from very different industries – including restaurants and comedy. And, along the way, I’ll experiment with new kinds of feedback loops.

In the coming weeks, I’ll post the project plan for feedback. I’ll share what I’m reading. I’ll do field work. I’ll put up conversations. Stay tuned for reflections from Chelsea Mauldin at Public Policy Lab in New York City and Chris Sigaloff and others at Kennisland in Amsterdam.

Do you have examples? Stories? Ideas? Critique? Please get in touch: hello@sarahschulman.com

Sarah Schulman

(this article was originally posted here: sarahschulman.com/the-nasty-questions)