“Release early, release often”… exactly to get feedback, which is seen as important. Here we are “releasing” lived-out experiences that translate into a call to action. The very first one is in the project proposal: visually impaired engineering student walks into wemake, would like to make some kind of kit for him to design/prototype circuits even with his poor eyesight, which makes soldering very hard. The solution itself is much less appealing, mobilizing and intuitive than the solution paired with the story.
Comfirm what Noemi says: in Berlin, the students were told: “You should post what you do right from the concept phase. Do it for two reasons: (1) because it’s part of the course and (2) because you are likely to get interesting feedback from the community, and it’s more fun to design for and with people than do it on your own. Think of it as another feedback channel.” And so they did.
Apparently we do not disagree on the part of releasing often, and which has already happened: Please take a look at the summary of the process that that the students went through. They were already sharing since round one, but in a 4 weeks workshop, and given the on boarding and the level of engagement t, how do you think the engamenet factor on ER could have been done better in a way that helps them progress, and helps opencare receieve more interesting ideas? We already believe in the value of sharing the story early, and having a conversation “starting somwhere”, as a “catalyst” to some idea development, my argument is, do we agree and acknowledge that sharing a story, the way it is done now, helps with some idea development and is bound to an organic time frame? If we agree, then do we think we need to define different engagement model that does a different job from the current possible catalyst one?
The process you point to seems reasonable to me. If it is true that people benefit from feedback, getting feedback from the ER community would also have been useful. 4 weeks is more than enough to do that. Also, there is no extra cost, because students need anyway to write what they want to do, why etc. But that is your call, really. You are welcome to run your engagement process as you see fit, and if you cannot make it work with ER, use something else. Costantino and I agreed this in Stockholm. As long as you produce ethno data in acceptable quantity and quality, you are delivering. See this comment below for more details.
As for your questions:
do we agree and acknowledge that sharing a story, the way it is done now, helps with some idea development and is bound to an organic time frame? Yes. Although I don't understand the "organic time frame" part. Conversation tends to be ongoing: this thread started by @Pauline during the UDK course in May is still active, though the course is long over.
do we think we need to define different engagement model that does a different job from the current possible catalyst one? My answer depends on what you would consider to be the current model.
If the current model is the Edgeryders one, no. It's working quite well for us. People engage. We get the data. Amelia codes them. Guy aggregates everything. The results make sense. We just need more of them (perhaps twice the volume) to make for great results.
If the current model is the one WeMake uses ofline or in contexts different from edgeryders.eu, I don't know. I have not seen what comes out of your process. Standards of data quality in opencare have been encoded in the Data Management Plan back in June and in Amelia's comments to this thread.
If the current model is what I am seeing on edgeryders.eu (posts by Wave, no reply to comments), then yes, we need a different engagement model. I think this is not generating enough data and of enough quality to justify the grant.
I wrote my other comment without refereshing, hence the cross post. A few points of clarification:
4 weeks are enough to have a conversation, but if ideation phase was bound to less than two weeks, then it is a bit tight (it was a marathon, in this particular case, as @Noemi said :))
Organic time frame, means that under no circmestances we can expect certain amount or directin of comments in a certain time frame. Some stories have been around for weeks without interaction, some were picked up quickly.
The current model, as I see it, is that we have an open call for stories, and we are inspiring people, mostly those we meet on the ground, to share their story around the notion of "care". Some problems do fit opencare context, some are too broad to be solved by one project or soluion (depression, global warming..etc). Our current focus is on harvesting a broad scheme of stories, and trying to get the conversation started (having 2-4 comments). We don't necessairly match a storyteller with a problem solver (because we don't have a call from problem solvers, like how we have a call for stories), and we don't have clear mechanisms to motivate the online storyteller to share their story (an online campaign vs on ground effort). I am not criticizing our existing model, I am analyzing it, and asking about, if/how we will change, moving forward. As I see, from your replies @Alberto, we want to maintain our existing model, while finding ways to harvest more stories, rather than think of applying differnet process, or motivations.
Yes, @Moushira , you are correct as far as I can see.
Add-on: in the ER methodology, the storyteller and the problem solver tend to be the same people (like in Nightscout). We call this asset mapping. The question is not “what is needed?”, but “what is already being done?”. You are very welcome, however, to do things differently and frame a call for problem solvers, if you think that is a good idea. You know your community well (just look at @zoescope 's unique experience!) and will have ideas on how to motivate it.
Once you have onboarded solution providers and they have somehow reacted (for example by providing a documented prototype) I would recommend to do online engagement being done the Edgeryders way. It’s hard work (it takes you leaving 800 comments to have done engagement) but it works; also, year 2 is not a time to experiment, but a time to consolidate. But this is just a recommendation. In the end it’s your budget, you answer for your own resuls, so it is your decision.
Lets just acknowledge the fact that we can’t work on separate ways. i.e. stories like this are great, but this is not a starting material for a solution or something to be matched with a maker. Thats why I earlier asked about how we see the stories moving forward besides archival – if we will continue to harvest stories, without a clear “criteria”, then at some point, we wil have data, but most of it, doesn’t fit a solution/product entry point. This is not about the number of comments we make in order to boost the conversation, in any case, 800 comments are not a problem, only if our comments are not going to be two thirds of the overall comments.
Assuming that we will move forward with the call for makers, is there a way we can have a break down of stories, categorized by theme of “seeking solution” vs “general story sharing”? That would be a good starting point for matching with makers. If not, we can think of alternatives.
Thanks for sharing your insights, am glad we are discussing details with transparency and dedication.
I want to resist the idea that there is a big difference between “objective” conversation and stories ---- I think they are not as different as we might think at first glance.
I think if we look at the kinds of things people have been posting, we find that there is more of a continuum than a direct difference between personal stories and conversations about clear outcomes. We have people sharing their experiences making something for themselves or their loved ones (like Night Scout), but we also have people talking about their organizations and the people who are benefitting from them (like Orange House), which is still a step removed from, for example, getting stories from refugees themselves. Yet I wouldn’t consider this any less of a personal experience. Similarly, students who have been given formal exercises to formulate their ideas, to me, are also valuable community members whose stories of creating technologies to deliver care are also important. From reading their posts, their contributions fit into the framework ---- presenting a problem, offering a solution. Then community members on Edgeryders interacting with them to ask for more information, make connections, etc. The important thing, from my perspective (and as @Alberto points out) is for them to stick around and respond to the comments. The value of sharing stories is to understand a) what kind of problems around care people are picking up on and addressing and b) how others react to their approach — do they share similar problems, are they problematising the way that they are approaching it and offering other solutions, do they think the questions need to be reframed, etc. These reactions can take place on the platform (this is ideal) or in the actual world (still good, but we need a way of documenting this, as you point out @Moushira ).
It depends on what we are interested in. If we are trying, as @Alberto is, to map networks of people talking to one another, it is not entirely ideal to have someone reporting someone else’s experience. If, though, we are going for a repository of knowledge and ideas, it doesn’t matter so much.
Once we get to talking on a more macro level about the purpose each of the posts serve (much like we are doing now) we get into slightly different terrain. This is higher-level analytical work we are doing (for example, @Moushira , the content of your comment). I am trying to group these kinds of discussions under a “methodology” tag.
Here would be my ideal:
Tell stories in the first person as much as possible. To capture information, we need people writing on the platform itself rather than linking to information elsewhere. So, as the students are doing, writing about their projects.
If someone has commented or interacted with you in the actual world, in my view it would be most helpful to record those in comments below the post. We are still thinking through how to do this, but creating dummy accounts would be a way of capturing interactions. Alternatively, if this is too difficult, it is okay to put it in the body of the post itself— we will capture the content (knowledge, ideas, reactions) but we will lose the interactive node element.
non-stories are fine too — analysis is really important, especially for writing reports, generating new and innovative methodologies, and making sense of data. We can gather these kinds of conversations and use them to make higher-level decisions about the project itself, what the data means, etc. I can still capture this information through codes and hierarchies.
Thanks @Amelia – much needed. In the coming phase of opencare you’ll be fairly central, because we do need the ethnographic data, and people need to know what consititutes “good” ethnographic data. opencare is funded by DG CNECT (formerly known as DG Information Society); more specifically, by the Collective Awareness Platforms Programme. This program funds research on how online platforms can be used to elicit collectively intelligent responses. Putting a project on a website does not qualify as collective intelligence: in the proposal, we have argued that online conversation around a project does. That is what we promised to do. Of course, discussion does not always catch on: that’s well understood. But on the whole, we should have enough conversation to be able to argue that ours is a really collective process.
I think that is defensible: we have 323 posts and 1,423 comments authored by 201 people, so an average of over four comments per post. 13 threads have 20 comments or more, two of them started by Berlin students. This translates into a social network with 201 nodes and 766 edges. The network’s topology suggests a dense conversation, with all 201 nodes part of the giant component. Today it looke like this:
It did not come for free: over 800 of the comments come from either Noemi, Nadia or myself. These are all questions, encouragement, feedback, requests for clarification. Without this kind of work, you cannot expect results.
Summarizing:
We need to produce high-quality ethnographic data. That means both stories and formal exercises.
It's important that proponents of either stick around to reply to comments and questions.
Engagement is done by having paid staff spend a lot of time on the platform to move the conversation along with prompts, thoughtful comments, encouragement and questions.
This does not have to be done on edgeryders.eu. In June, @Costantino and I agreed that WeMake is free to use other platforms if that works better for you. In this case you will have to do what we have done for edgeryders.eu:
Cover the ethical issues by installing a consent funnel or equivalent.
Secure legal clearance to reuse the content (IPR on uploaded content).
Organize a workflow so that the content can be coded by an ethnographer, without losing the contextual information of who authored the coded content, in which context, talking to whom, etc.
Secure machine-readable access to the primary data (content) and the secondary data (ethnographic coding), so that @melancon can import them in his dashboard. We can demo the dashboard in Milano, already with real data.
One final other comment ---- the other important thing is that posts have a certain level of detail. The main problem I see with some of the student posts is they don’t have quite enough information in them for people to be able to interact well. As @Noemi mentions in her comments on one of the posts, including a call to action, asking questions, or suggesting points of intervention allows community members to interact more with the post. Framing the narrative as a “story” is less important than offering a certain amount of information about process. And it doesn’t have to be too much more, but more is definitely needed.
One the value of the story: From a design or product perspective, without a story, it is nearly impossible to progress, because then there would be no clear problem to solve. The one minor concern here is that writing is a skill, so not everyone is able to craft a catchy story (in their second language) and as a student, the influence of presenting research and work, can mask the empathy factor in the story, even if the story is quite personal. This can happen with any formal assignment and imo, there is no problem with that.
@Alberto thanks for sharing the data. The qantitative approach is interesting, according to the numbers: Paid stuff are doing nearly 60% of comments, then a) do we need to target different audiance? (match storyteller with a problem solver) b) do we need to explore a new process?, if no, then, do we need to discuss the future of the repository of stories, besides archival? Needless to say, storytelling is a leading and trending tool in design, we can not undermine the power of a story, and we can’t claim that a story alone isn’t important, but we can still question, which direction do we want the story to serve, besides contributing to our diverse repository.
We need to keep questioning our processes as we move on, it is not a sign of conflict at all, it is a sign of care for what we do and passion for getting the most to our global community and world problems, from the tools and mechanisims we use. Thanks, and nice to meet you @Amelia, btw
I understand your point about writing being a skill and the students not feeling confident in sharing in a second language. However, i would suggest that the process of writing about the project is more important than the literary value of the content. If you look at the posts below you see a story developing between 3 members of a design team and the community.
At each stage the community engages to some degree, initailly to say hello and encourage development and thought.
By the final post there is a high level of engagement, including other ideas, suggestions and development possibilities. We also start to see where the project might go next.
A few students also didn['t feel comfortable writing in English so chose to include content in German. We translate to the best of our abilities and then ask questions seeking clarification.
Again, apparently we don’t disagree :). The level of narrative in the stories the student put together isn’t bad, imo, and it falls along the lines of the engagement model here, I guess: https://edgeryders.eu/en/not-a-clash-of-culture-but-a-merge-of-culture it could of course have been made different, but has nothing to do feedback, imo. I was just acknolwledging the fact that different stories have different motivations and styles. The refugee story has been around for long, has an ongoing on the ground plan, hence, imo, the different number of comments.
Great examples, @Alex_Levene ! Thanks for prompting me to re-read these stories in sequence. You are absolutely right. It’s quite amazing as collective problem solving goes: it starts with “stubs” of “why are people in Germany afraid of migrants?”, and three posts later you get @trythis (an honest-to God material scientist) getting in touch about the many great things you can do with epoxy in refugee camps, and where you might get it for cheap in Germany. You have gone from abstract to fully technical, and have done so in a group of people who have never met in real life, nor are likely to.
Two methodological notes:
Everyone: @Alex_Levene himself is a recipient of one of the opencare fellowships. Impressed by a post he wrote, we asked him to write a new one and rewarded him with gratitude and a small sum of money. So, @Costantino , this is a motivating move that you could copy: make it clear that you will reward small contributions.
The examples of @Tomma and the team with @MarieScheurer are great also because these are all people who speak English as a foreign language. Their expression is English is not perfect (and neither is mine), yet it's good enough for engagement.
One meta-methodological issue: @Amelia , how is this great thread by @Moushira for meta-ethnography? Does it qualify?
Many comments by staff are not a problem at all. Reason: they typically do not get coded by Amelia anyway. They are in large part encouragements and requests for clarifications. They are a major part of the motivating mechanism that @Costantino is searching for. So, the ethno analysis still represents what the community thinks and does – and there are hundreds of comments written by the community.
Pro tip: do not think in shares (60% of the comments are written by the staff!). Think in absolute numbers (the community has written 600 comments and 300 posts!). As with any data problem, discarding stuff is fairly easy, whereas getting it is expensive (and slow). If you can bring Guy 500 more comments and 1-200 posts, and ScImpulse can do the same, we are good.
UPDATE – I checked again my script, and found there was some double counting in team comments (really it’s a MySQL known issue, that you can get rid of via code). The data as of now are:
We should remember that a large amount (I would venture to say 60% ) of ethnographic data is generated through interviews with research participants. And what are interviews if not “encouragements and requests for clarifications”? The ethnographic process assumes a person (or in our case, people) asking participants more in-depth questions, prompting them to think more deeply about things they might take for granted as parts of their daily process. If we are doing collective ethnography, I see the work that the team does as important ethnographic work.
Also on the subject of stories… I was reminded of this quote from anthropologist Michael Fischer (in a 2015 American Ethnologist article):
“Stories are not just stories of, but stories for. They are ways of thinking about how we want to build communities of the future, for ourselves as well as others…they provide the mental tools as well for sur-vie”
Thanks for the updates, I just got lost in the last scentence: “If you can bring Guy 500 more comments and 1-200 posts, and ScImpulse can do the same, we are good.” What does that mean?
Data is a very powerful and (and very flexible) indicator, indeed. Talking earlier about meta topics, I am drafting another separate post for some questions around how we outline our lessons learned. Thanks again!
Sorry for cloudy writing. It means this: Guy and his group at UBx are aggregating ethno data into and crunching it. For now, they have the 322 posts and 1,442 comments (of which 840 written by community members – the numbers keep changing as new material rolls in) on edgeryders.eu. Most of these (almost all the comments) are the result of engagement done by Edgeryders.
What I am saying is: WeMake and ScImpulse should each do engagement activities that result in 500 community-authored comments of reasonably high quality. In this way, we would have roughly 800 + 500 + 500 = 1,800 community authored comments, perhaps 3,000 comments overall, and we would be good. “Good” means we have enough high-quality content to mount a large scale, network powered ethnography and see if (a) our method works at scale; and (b) how communities go about inventing and deploying care services. This is consistent with the added value of our project (section 1.3.6 of the proposal).
You can execute on our methods, or try something else. You can do it on edgeryders.eu, or anywhere. The comments could be reaction to posts, like here, or to schematics or videos of instructables, as you prefer. But that. I would say, is your goal (see also section 1.1.3 of the proposal).
) of ethnographic data is generated through interviews with research participants. And what are interviews if not “encouragements and requests for clarifications”? The ethnographic process assumes a person (or in our case, people) asking participants more in-depth questions, prompting them to think more deeply about things they might take for granted as parts of their daily process. If we are doing collective ethnography, I see the work that the team does as important ethnographic work.