Engineers Without Borders (https://www.ewb.ca) is an international organization dedicated to creating systemic and sustainable change. It is committed to addressing the root causes of poverty by investing in Canadian and Sub-Saharan ventures and propelling social innovation forward.
EWB has always been eager to learn from its shortcomings. Following the launch of its Admitting Failure Venture in 2008, it has continued to publicize its organizational failures in its Annual Failure Report (reports.ewb.ca). Its attempt to model an attitude of constructive learning is unparalleled in the international development space.
But this publication process is only one part of the learning cycle. Our experimental learning approach must be intentional. We must design clear frameworks, define measurable hypotheses, and use meaningful metrics to ensure that our failures are translated into actions. In order for our learnings to have depth, direction, and substance, we need to treat failing forward as a way of line instead of a deadline.
This session will adopt a critical lens in discussing the following:
-An exploration of the failure learning cycle and why it’s broken in development
-EWB’s past 8 year history with producing its national failure report
-Engaging donors and other stakeholders in the conversation
-Building authenticity in admitting failure
-How do we publish stories that matter?
-How do we measure the impact of the report? Is the report meeting a need? Who are our target audiences?
-How do we translate learnings into actions? And, if we are repeating failures, how do we restructure this process?
@Christine_Pu thank you. I have to ask because this is an important statement, so burning with curiosity. What do you mean by that, can you give us a short preview? Is that something you learned from your own organisations’ failures or from the stakeholders you work with e.g. clients of Admitting Failure?
Context: In the community call we had earlier today, @trythis was cautioning us against an event that parades failure, and encouraging us to find stories backed by data and metrics. Your session seems to go a long way to achieve this and you are one of the few professionals on failure who are attending, so thank you so much in advance
Fantastic question! I am by no means an expert, but just from my experience with working on EWB’s failure report, I would propose that the cycle is broken at the first step - admitting failure. Our development organizations prioritize the influence of their donors over the needs of the communities they intend to serve. We apply incorrect assumptions to foreign circumstances, unintentionally oversimplify situations, and resort to plans of action that have only failed us in the past. But due to the donor-driven nature of development, it makes more sense financially to hide failures than own up to them. And this doesn’t just hold for donors either. As a society, we CARE about reputation and we define success metrics to be extremely outcome-oriented. There is nothing wrong with this, but it does perpetuate the negative connotation of failure. As a result, learnings are not being shared nor internalized, and NGOs are not moving through the learning cycle. This is likely to lead to a bottleneck of repeated mistakes.
Even if you are an organisation willing to try out things and say from the start “let’s test this and see how it goes before we assume our method can’t fail”, well that’s not something funders would fund often. And this is not just specific to development: because donors everywhere are accountable at other levels too, it seems they have a hard time upselling a “tryout” phase, then re-assess, then fund a bigger chunk of work. What’s more, this makes it dangerous for your organisation too, starting something and then not knowing for sure you will be able to continue good work just when results start to show.
With Edgeryders we are learning from this and it’s one of the reasons we began to question the effectiveness of short term community building processes in under-/less developed areas. @Matthias’s reflections after doing work in post-earthquake Nepal are superb and disheartening at the same time, but made a real impact in how we will approach such projects. So this would be internalizing in a way. You could say this is a public stance too since we often talk here on an open forum, but we are far from metric based assessment per se.
Since you mentioned my reflections on development in Nepal, I just re-read what I had nearly forgotten existed Here are some additions after some months of intensive thinking about development (and Nepal in particular):
With respect to metrics: they are great where you can use them, but it’s less often the case than one might want. Because they only make sense when you keep trying what you tried before, as otherwise you need other metrics. However failure in development is more often than not hinting that you might be trying the wrong thing. As a simplistic example, building strong social movements and civil society in Nepal is the wrong thing at the current time, as the first thing to deal with would be enabling a decent surplus in the rural economy that will grant villagers the time needed to engage deeply into civil society building. So metrics of movement building would be abandoned when understanding this …
In the reflections that Noemi linked to, I argued for introducing network bartering in rural Nepal to grant some people a “full-time equivalent job” in movement building where monetary funding is not sufficient for that. I abandoned that idea, based on first-hand experiences in our project about how difficult introducing a digital platform is even in urban centers in Nepal. So, I learned from failure, right? Yes, seems like. But it won’t be obvious, since I’ll not publish the 60-80 ideas I abandoned in idea stage, and will instead fail differently with the next idea I think is great.
So the question remains, why won’t I get it right about ideas for development? Why don’t I simply learn from the failure of others? In my current view, because that’s not “simple”. It’s not the stuff you can learn from books: you can’t use a theory, but you need a truckload of experiences and memories to be able to explore in your mind if a certain idea would work in a certain development context. A theory only really helps to solve problems if you can use it for calculations. If we can ultimately have that for development, fine, but until then: How about “immersive failure experiences” – invite people to join the late stages of failing projects where it’s about talking to locals, understanding the failure, and wrapping up. That’s the brain fodder on which to build own expertise, without having to do all the work to create a failing project on your own
Expertise in a development context means understanding the people you want to serve. How their mind works. Finally, close to five months after leaving Nepal, I made progress in trying to understand village people in Nepal. What helped? Talking to a friend who grew up in a small village in Nepal and made the transition to the West, now living in a major city for over a decade. She’s expert in both cultures and could finally answer my questions. This would not have worked by me asking a local person in a village in Nepal. It needs somebody who became the expert by being exposed for a long time to both cultures, allowing to build a mental model of both from inside and outside of both cultures.
Deep stuff in there, @Matthias. I guess we will be speaking about this for a long time .
Just one note that I cannot resist adding right now: development work is about reproducing artificially what happened spontaneously somewhere else (eg. Southern Germany, or the Silicon Valley). This means introducing an element of design and command-and-control to drive a complex system in a way that will mimic the emergent behaviour you like. “Design for emergence” is a paradox, of course, and has many consequences.
Your notion of sequencing (agricultural surplus first, then civil society development) is closer to engineering than the biological metaphors that I prefer to employ when I think about this stuff. Engineering metaphors make things appear simpler than they are. For example, the insistence on civil society development, in this space, comes from the experience of putting a lot of money into creating initiatives that were captured by powerful vested interests leaning on weak, corrupt, unaccountable governments. So the “right sequence” was wrong, too. This is because, when you are trying to build stuff in the social world there is very likely no right sequence: it’s chicken-and-egg problems all the way down.
This may sound paralysing, but it is actually not. Reason: you can employ the powerful concept of subgame perfect equilibrium (formal definition). The intuition is: each step of the way makes sense in and of itself, without needing to be underpinned by the final grand goal. Father Cassian Folsom’s critique of the unMonastery (this will not work without God) can be interpreted in that way: a monk’s avocation is subgame-perfect, because as they learn to live in a community in relative peace (which is really hard), they get immediate reward by serving God. On the other hand, a lay person trying to do the same in an unMonastery will have to motivate herself without such immediate reward, until the community is finally stable. So, says Father Cassian (translated in the language of game theory), the unMonastic path is not subgame-perfect.
In development, sequences need not be “right”. They need to be subgame-perfect.
@Christine_Pu this should be a good snapshot of what you can scratch with this session and the kind of experiences some people in the crowd have. Hope you can use some of it as you prepare your session. I would expect our interests to go way beyond the how to’s. From this conversation, it is clear that inside the same organisation people have different takes on what counts as failure. The most one can do is talk about challenges.
We’re also in the business of community building, where cutoff points for metrics are very hard to pin down. @Matthias, for what it’s worth I’m always trying to remember that and not beat myself up: community building is not the same as economy building (although after Bucharest I also ask myself: but what have we changed to make people’s lives better?)
And that could be true, Alberto! It’s important for us to remember that admitting and publicizing failure really is only ONE step in this learning cycle. We need to be intentional about the way we set up our experimental learning processes. We need to define our hypotheses and meaningful metrics before we draw conclusions on them. And it’s equally important to determine how best to translate our learnings into action!
Flickr was the result of a failed game design, but the communication section of the design was turned into Flickr ands they ditched the main thing they got funded for. A good example of adroitly moving sideways with the part that does work.
Thanks for sharing! That is type of “learning” we are hoping to accomplish through publicizing our failures…and “we” could refer to the entire international development sector itself!
.
