Lets just acknowledge the fact that we can’t work on separate ways. i.e. stories like this are great, but this is not a starting material for a solution or something to be matched with a maker. Thats why I earlier asked about how we see the stories moving forward besides archival – if we will continue to harvest stories, without a clear “criteria”, then at some point, we wil have data, but most of it, doesn’t fit a solution/product entry point. This is not about the number of comments we make in order to boost the conversation, in any case, 800 comments are not a problem, only if our comments are not going to be two thirds of the overall comments.
Assuming that we will move forward with the call for makers, is there a way we can have a break down of stories, categorized by theme of “seeking solution” vs “general story sharing”? That would be a good starting point for matching with makers. If not, we can think of alternatives.
Thanks for sharing your insights, am glad we are discussing details with transparency and dedication.
I want to resist the idea that there is a big difference between “objective” conversation and stories ---- I think they are not as different as we might think at first glance.
I think if we look at the kinds of things people have been posting, we find that there is more of a continuum than a direct difference between personal stories and conversations about clear outcomes. We have people sharing their experiences making something for themselves or their loved ones (like Night Scout), but we also have people talking about their organizations and the people who are benefitting from them (like Orange House), which is still a step removed from, for example, getting stories from refugees themselves. Yet I wouldn’t consider this any less of a personal experience. Similarly, students who have been given formal exercises to formulate their ideas, to me, are also valuable community members whose stories of creating technologies to deliver care are also important. From reading their posts, their contributions fit into the framework ---- presenting a problem, offering a solution. Then community members on Edgeryders interacting with them to ask for more information, make connections, etc. The important thing, from my perspective (and as @Alberto points out) is for them to stick around and respond to the comments. The value of sharing stories is to understand a) what kind of problems around care people are picking up on and addressing and b) how others react to their approach — do they share similar problems, are they problematising the way that they are approaching it and offering other solutions, do they think the questions need to be reframed, etc. These reactions can take place on the platform (this is ideal) or in the actual world (still good, but we need a way of documenting this, as you point out @Moushira ).
It depends on what we are interested in. If we are trying, as @Alberto is, to map networks of people talking to one another, it is not entirely ideal to have someone reporting someone else’s experience. If, though, we are going for a repository of knowledge and ideas, it doesn’t matter so much.
Once we get to talking on a more macro level about the purpose each of the posts serve (much like we are doing now) we get into slightly different terrain. This is higher-level analytical work we are doing (for example, @Moushira , the content of your comment). I am trying to group these kinds of discussions under a “methodology” tag.
Here would be my ideal:
Tell stories in the first person as much as possible. To capture information, we need people writing on the platform itself rather than linking to information elsewhere. So, as the students are doing, writing about their projects.
If someone has commented or interacted with you in the actual world, in my view it would be most helpful to record those in comments below the post. We are still thinking through how to do this, but creating dummy accounts would be a way of capturing interactions. Alternatively, if this is too difficult, it is okay to put it in the body of the post itself— we will capture the content (knowledge, ideas, reactions) but we will lose the interactive node element.
non-stories are fine too — analysis is really important, especially for writing reports, generating new and innovative methodologies, and making sense of data. We can gather these kinds of conversations and use them to make higher-level decisions about the project itself, what the data means, etc. I can still capture this information through codes and hierarchies.
Thanks @Amelia – much needed. In the coming phase of opencare you’ll be fairly central, because we do need the ethnographic data, and people need to know what consititutes “good” ethnographic data. opencare is funded by DG CNECT (formerly known as DG Information Society); more specifically, by the Collective Awareness Platforms Programme. This program funds research on how online platforms can be used to elicit collectively intelligent responses. Putting a project on a website does not qualify as collective intelligence: in the proposal, we have argued that online conversation around a project does. That is what we promised to do. Of course, discussion does not always catch on: that’s well understood. But on the whole, we should have enough conversation to be able to argue that ours is a really collective process.
I think that is defensible: we have 323 posts and 1,423 comments authored by 201 people, so an average of over four comments per post. 13 threads have 20 comments or more, two of them started by Berlin students. This translates into a social network with 201 nodes and 766 edges. The network’s topology suggests a dense conversation, with all 201 nodes part of the giant component. Today it looke like this:
It did not come for free: over 800 of the comments come from either Noemi, Nadia or myself. These are all questions, encouragement, feedback, requests for clarification. Without this kind of work, you cannot expect results.
Summarizing:
We need to produce high-quality ethnographic data. That means both stories and formal exercises.
It's important that proponents of either stick around to reply to comments and questions.
Engagement is done by having paid staff spend a lot of time on the platform to move the conversation along with prompts, thoughtful comments, encouragement and questions.
This does not have to be done on edgeryders.eu. In June, @Costantino and I agreed that WeMake is free to use other platforms if that works better for you. In this case you will have to do what we have done for edgeryders.eu:
Cover the ethical issues by installing a consent funnel or equivalent.
Secure legal clearance to reuse the content (IPR on uploaded content).
Organize a workflow so that the content can be coded by an ethnographer, without losing the contextual information of who authored the coded content, in which context, talking to whom, etc.
Secure machine-readable access to the primary data (content) and the secondary data (ethnographic coding), so that @melancon can import them in his dashboard. We can demo the dashboard in Milano, already with real data.
One final other comment ---- the other important thing is that posts have a certain level of detail. The main problem I see with some of the student posts is they don’t have quite enough information in them for people to be able to interact well. As @Noemi mentions in her comments on one of the posts, including a call to action, asking questions, or suggesting points of intervention allows community members to interact more with the post. Framing the narrative as a “story” is less important than offering a certain amount of information about process. And it doesn’t have to be too much more, but more is definitely needed.
One the value of the story: From a design or product perspective, without a story, it is nearly impossible to progress, because then there would be no clear problem to solve. The one minor concern here is that writing is a skill, so not everyone is able to craft a catchy story (in their second language) and as a student, the influence of presenting research and work, can mask the empathy factor in the story, even if the story is quite personal. This can happen with any formal assignment and imo, there is no problem with that.
@Alberto thanks for sharing the data. The qantitative approach is interesting, according to the numbers: Paid stuff are doing nearly 60% of comments, then a) do we need to target different audiance? (match storyteller with a problem solver) b) do we need to explore a new process?, if no, then, do we need to discuss the future of the repository of stories, besides archival? Needless to say, storytelling is a leading and trending tool in design, we can not undermine the power of a story, and we can’t claim that a story alone isn’t important, but we can still question, which direction do we want the story to serve, besides contributing to our diverse repository.
We need to keep questioning our processes as we move on, it is not a sign of conflict at all, it is a sign of care for what we do and passion for getting the most to our global community and world problems, from the tools and mechanisims we use. Thanks, and nice to meet you @Amelia, btw
I understand your point about writing being a skill and the students not feeling confident in sharing in a second language. However, i would suggest that the process of writing about the project is more important than the literary value of the content. If you look at the posts below you see a story developing between 3 members of a design team and the community.
At each stage the community engages to some degree, initailly to say hello and encourage development and thought.
By the final post there is a high level of engagement, including other ideas, suggestions and development possibilities. We also start to see where the project might go next.
A few students also didn['t feel comfortable writing in English so chose to include content in German. We translate to the best of our abilities and then ask questions seeking clarification.
Again, apparently we don’t disagree :). The level of narrative in the stories the student put together isn’t bad, imo, and it falls along the lines of the engagement model here, I guess: https://edgeryders.eu/en/not-a-clash-of-culture-but-a-merge-of-culture it could of course have been made different, but has nothing to do feedback, imo. I was just acknolwledging the fact that different stories have different motivations and styles. The refugee story has been around for long, has an ongoing on the ground plan, hence, imo, the different number of comments.
Great examples, @Alex_Levene ! Thanks for prompting me to re-read these stories in sequence. You are absolutely right. It’s quite amazing as collective problem solving goes: it starts with “stubs” of “why are people in Germany afraid of migrants?”, and three posts later you get @trythis (an honest-to God material scientist) getting in touch about the many great things you can do with epoxy in refugee camps, and where you might get it for cheap in Germany. You have gone from abstract to fully technical, and have done so in a group of people who have never met in real life, nor are likely to.
Two methodological notes:
Everyone: @Alex_Levene himself is a recipient of one of the opencare fellowships. Impressed by a post he wrote, we asked him to write a new one and rewarded him with gratitude and a small sum of money. So, @Costantino , this is a motivating move that you could copy: make it clear that you will reward small contributions.
The examples of @Tomma and the team with @MarieScheurer are great also because these are all people who speak English as a foreign language. Their expression is English is not perfect (and neither is mine), yet it's good enough for engagement.
One meta-methodological issue: @Amelia , how is this great thread by @Moushira for meta-ethnography? Does it qualify?
Many comments by staff are not a problem at all. Reason: they typically do not get coded by Amelia anyway. They are in large part encouragements and requests for clarifications. They are a major part of the motivating mechanism that @Costantino is searching for. So, the ethno analysis still represents what the community thinks and does – and there are hundreds of comments written by the community.
Pro tip: do not think in shares (60% of the comments are written by the staff!). Think in absolute numbers (the community has written 600 comments and 300 posts!). As with any data problem, discarding stuff is fairly easy, whereas getting it is expensive (and slow). If you can bring Guy 500 more comments and 1-200 posts, and ScImpulse can do the same, we are good.
UPDATE – I checked again my script, and found there was some double counting in team comments (really it’s a MySQL known issue, that you can get rid of via code). The data as of now are:
We should remember that a large amount (I would venture to say 60% ) of ethnographic data is generated through interviews with research participants. And what are interviews if not “encouragements and requests for clarifications”? The ethnographic process assumes a person (or in our case, people) asking participants more in-depth questions, prompting them to think more deeply about things they might take for granted as parts of their daily process. If we are doing collective ethnography, I see the work that the team does as important ethnographic work.
Also on the subject of stories… I was reminded of this quote from anthropologist Michael Fischer (in a 2015 American Ethnologist article):
“Stories are not just stories of, but stories for. They are ways of thinking about how we want to build communities of the future, for ourselves as well as others…they provide the mental tools as well for sur-vie”
Thanks for the updates, I just got lost in the last scentence: “If you can bring Guy 500 more comments and 1-200 posts, and ScImpulse can do the same, we are good.” What does that mean?
Data is a very powerful and (and very flexible) indicator, indeed. Talking earlier about meta topics, I am drafting another separate post for some questions around how we outline our lessons learned. Thanks again!
Sorry for cloudy writing. It means this: Guy and his group at UBx are aggregating ethno data into and crunching it. For now, they have the 322 posts and 1,442 comments (of which 840 written by community members – the numbers keep changing as new material rolls in) on edgeryders.eu. Most of these (almost all the comments) are the result of engagement done by Edgeryders.
What I am saying is: WeMake and ScImpulse should each do engagement activities that result in 500 community-authored comments of reasonably high quality. In this way, we would have roughly 800 + 500 + 500 = 1,800 community authored comments, perhaps 3,000 comments overall, and we would be good. “Good” means we have enough high-quality content to mount a large scale, network powered ethnography and see if (a) our method works at scale; and (b) how communities go about inventing and deploying care services. This is consistent with the added value of our project (section 1.3.6 of the proposal).
You can execute on our methods, or try something else. You can do it on edgeryders.eu, or anywhere. The comments could be reaction to posts, like here, or to schematics or videos of instructables, as you prefer. But that. I would say, is your goal (see also section 1.1.3 of the proposal).
Thanks @Alberto for the update, just wondering about any qualitative criteria that we have besides the numbers? As with the approach of doing activities on the ground then transferring the stories narratievly on ER, where 80% of those will be written by staff, and not surprisngly a good chunck of comments will be by staff as well. As long as the number is sufficient, then the metric is still OK? There are some cases studies around openIDEO, with a qualitative analysis that involved interviews. I wonder if a brief booklet analysing the experient on ER will be compiled at some point? Thanks!
“Putting a project on a website does not qualify as collective intelligence: in the proposal, we have argued that online conversation around a project does.”
We all agree of the value of the conversation that can be “exploited” while is online and codable as in ER. Consider this well bought by me.
To me the central point is this:
how do we REALLY motivate people to overcome the friction in the process of writing on line?? What’s the gain in writing here on ER or writing on line in general? Of course these are not question for a black or white answer. We need to discover and levarage on the trade-off: as more value I can fell/exctrat as more “pain”/friction/effort I can accept.
Concerning some experiments:
if we have students (as in this case) we can give them "homework" and also in this case is not smooth AT ALL
when we don't have students the "homework" apporoach is not feasible.
as previoulsy said the OpenANDChange strategy is not working in a design/making environment mainly because the "stuff to be produced" is not a document; if the subject of your development is a doc -> you're writing the doc -> you're already writing -> it's the same environment / approach
So we need to have other working strategies (and than protocols and tools and how to …) that consider other human approach and use cases. Just to mentions some cases:
- when the product of the process are no docs file
- when the second language is an issue/barrier
- when the platform is not frictionless
- when a human is not "verbose" type
- when the value of writing is not exaclty valuable for the "supposed to be" the user
- when the process of writing -> gaining value it's not clear or recognized from a user.
when the value of this "collective intelligence" is hidden and not available for the user
@Alberto IMHO this is what we need to takle in order to demonstrate not only that is possible to analyze the “data” when they are available, but we also have sustainable way to create the right conditions to foster the users creating these shared value, maybe considering this kind of approach for the whole process.
Self-selection, engagement and a pinch of innovation
Yes. The “what’s in it for me” question is, as always, the central one of engagement.
Your list of use cases is, as far as I can see, fundamentally splittable into two parts.
On case 1, I have no answer. Your experience is much deeper than that of any of us in the consortium. If you can’t solve it, no one can. Zoe is almost certainly in the worldwide top 10 experts on this issue. We go back to the Commission and say “not happening. Collective awareness platforms are not possible when the collective process is aimed at producing something other than a document.” It is a dignified negative result, still a result.
We have, however, met cases 2-7 many times. I have done my best to share this experience with you all, many times, but it seems it is so counterintuitive that you cannot bring yourselves to try it I think this is why Guy finds this conversation theoretical: practices that (sort of) work are already there, generating the data for this project, in exactly this context. Also, I am sure there are others. And yet, here we are, with 10 months at the most to wrap up the data generation phase, asking ourselves how to do engagement…
So, how do you want to proceed? How can we help? Shall I ask Noemi to give you a how-to presentation, maybe contextualized to the opencare experience? @johncoate gave one in LOTE5, but at the time opencare was just starting.
I am a bit worred, because it was our main promise that few people get this stuff, and we are among these few. This case is made in Section 1.3.2 of the proposal (Approach). The key passage is this:
We are also aware that, while successes get a lot of exposure, most attempts at mobilizing communities towards collectively intelligent outcomes fail.
OpenCare moves from the hypothesis that many unsuccessful outcomes are predicated on bad execution, in turn often caused by theoretical misunderstanding. Few would dispute that a smart community runs not only on ICT technology and infrastructure, but also on a web of correctly aligned incentives, shared values, social conventions, and common vocabulary. However, we propose that, in practice, most of the attention and resources of people and organizations wishing to convene smart communities are concentrated solely on ICT artefacts. The result is, unsurprisingly, empty platforms that host little meaningful conversation (Prieto-Martin et. al. 2012).
On one side, I do agree with you, that users may refrain posting on a platform, even more if they’re asked to do so after an activity has ended. Earlier comments underline the usefulness of having this takeplace while the process develops. The language barrier might not be that a barrier, assuming the content can be ethno coded from any language (usgin multilingual tags, let’s say).
Bottom line, users must of course see the added value of posting online, otherwise it may indeed be quite hard to have them spend the necessary effort to do so.
I also agree that we could/should develop other working strategies to get the material we are after.
–
On the other, more pragmatic, side, we need to make sure we deliver what we promised to deliver. Looking at things from the different tasks UBx is involved in, it is true we can only put our methods and tools at work if we can harvest some data echoing the excnahges between people. Stories or conversations, I have no firm opinion on this question, as long as the content can be tagged I’ll be fine
–
What I hear is that this thread oscillates between these two sides, looking at the issue from a “theoretical” viewpoint; or from a more pragmatic DG CNECT viewpoint keeping in mind we are bound to project deliverables.
–
I might be wrong, I count on you guys to help me get a better understanding of this issue.
) of ethnographic data is generated through interviews with research participants. And what are interviews if not “encouragements and requests for clarifications”? The ethnographic process assumes a person (or in our case, people) asking participants more in-depth questions, prompting them to think more deeply about things they might take for granted as parts of their daily process. If we are doing collective ethnography, I see the work that the team does as important ethnographic work.