Ethnography Team: Reflection on a Surveillance Pandemic

I’m posting our (@amelia @CCS @Leonie) Methods Reflections below:

Ethnography Team: Reflection on a Surveillance Pandemic

Section 1. Setting up the event in future

  1. Identify a topic for the event based upon the ethnographic analysis done so far. Ask: what topics are the community most interested in? Which would be most useful/generative to do a deep dive into?
  2. Have at least a month to plan the call, invite a diverse range of participants (including the most active community members on platform on this topic), and put informed consent procedures in place.
  3. Use the SSNA/codes/ethnography to pull out questions to ask participants during the call. (Research team can be present and ask questions as well as moderators).
  4. Record the call and transcribe the recording.
  5. Post the transcription and code the transcription.
  6. Generate an SSNA based upon it (we need to brainstorm ways to have the Social part of the SSNA visualised in this, but at worst it’s a visualisation of the main themes of the conversation. At best, we separate each contribution into a different comment to see what concepts most frequently co-occur together).

Section 2: Ethical consent funnel and procedures

Create an information sheet to put on the event page that is very easy to locate and is in clear, human language. This should also be present before the Zoom link is reachable/above the Zoom link. This needs to be done at least a week before the event, but preferably as soon as possible. This information sheet needs to contain at a minimum the following:

  1. Description of Research Project (what the data will be used for).
  2. Description of Event - duration, type of event, what they can expect to encounter.
  3. That the event is being recorded.
  4. How the event data will be made public, processed, and stored, and for how long (include the possibility that the call will be transcribed and posted).
  5. That they have the right and ability to withdraw at any point (up until the publication of final research results, give date).
  6. Who they can contact if they want any of their contributions removed/want to withdraw.
  7. Potential risks and benefits of participation (in this section, we must list that we cannot guarantee them anonymity or data privacy, in part because we are using an insecure video client)
  8. What body is overseeing the research: if ERC/H2020, for example, then mention that an ethics committee with ethics advisors is overseeing the project, and give their contact information.
  9. That they also agree to abide by these rules in terms of data protection and circulation – they are not permitted to make information public beyond what the research project allows in point 4.

Get participants to digitally sign an informed consent form with the above information at some point before they enter the call.

  • My thoughts on best practice for this: create an online consent form through something like Qualtrics or Google Forms (not sure about the security of the latter, would have to look into it-- or generate own internal form if we use the ER platform to video call). When the user submits the form, they are given the link to the video call and can enter. That way you know no one in the call has not gone through the consent funnel.
  • Example of an info sheet/consent form currently being used: https://www.smarten.org.uk/covid-19-study.html
  • Noemi’s “Meeting Registration” page would indeed work, but only if the I’m not a robot box was changed to a “I consent” box.
  • At the beginning of the call, remind people that the call is being recorded and of the information on the information sheet (particularly on their ability to withdraw/opt-out).
  • Inform all participants when the data is posted on the platform. Underneath/on the same post as the recording, give contact information for who to contact if they want to change their consent status and reiterate that they can do so.
  • Host the call on a secure platform so we can assure participants of a higher degree of data privacy/lower risk of data breach.

We also need to be clear about what the participants are allowed to use the data for. It was clear that there were a few journalists on the call looking for a scoop/for information to publish. We need to be clear that part of the agreement is that you as a participant can’t go off and write using any kind of identifiable data.

Section 3: Content and demographics:

A. Content is shaped by WHO is participating: notes on participant demographics and its implications for content

  1. Networks and recruiting participants:
    The virtual meeting consisted of 36 participants including experts within the legal, medical and public health, digital tech, privacy and human rights, public policy and media communities. Participants in the call were invited through existing social and professional networks, i.e. were largely existing contacts of the Edgeryders team, primarily Alberto (friends-of-friends approach). Thus, many were based in Italy, some in the UK, Sweden, Belgium and France. Expanding and diversifying our recruitment for future events will allow us to hear from a broader range of experts, researchers and practitioners. We should brainstorm ways to do this effectively, perhaps with @noemi and @nadia (and others experienced in outreach and engagement).

  2. Encouraging Participation:
    That being said, a full demographic overview is limited by the small number of introductions given by the participants. We should think of ways to give speakers the opportunity to introduce themselves (if they would like to/are able to) – in a manner that is comfortable to them. The chat was one useful way of doing this, but it’s not clear that chat introductions lead to more engagement between people of similar interests, since they came quickly and densely at the beginning.

In virtual events lately, @amelia has been doing a virtual version of the 15 minutes for coffee built in to schedules before conferences formally commence. One way of getting people more comfortable and chatting might be to schedule 15 minutes before the call as a “coffee and chat” period, a soft entry. This happened point to a certain extent, and could be further encouraged by making it an actual thing on the schedule.

This kind of space for introductions (or thinking of other ways to do intros) may also encourage participants to become acquainted with one another before the discussion begins, and might foster more interaction across a larger number of participants during the call. Of the 36 participants, we find a small handful of experts on call participating more than once/speaking more than once. The conversation was largely driven by 4-5 main voices + mediated by community managers.

The same goes for the conversation in the thread following Alberto’s report on the meeting. How do we maximize participation and encourage participants in calls to engage on the platform after the call? Also a topic we can brainstorm together.

  1. Making Discussions more Representative:
    The following report is based on the exchange between a small number of speakers (in the discussion and chat), and though there are some salient themes, this may not be representative of the concerns/ideas of the broader community.

To that end, the participants (or at least those who were the most active) represented ca 3-4 European countries (incl. the UK). How do we include a broader range of people in these discussions? In terms of empirical examples, we mainly focussed our discussion on the UK and Italy, two of the hardest-hit countries currently affected by the virus. We need to bring in cases from countries with different experiences/ cases where it’s ‘going well’: e.g. Germany, Finland, the Czech Republic etc.

This could be helped in our next call by pulling out questions we want to ask beforehand based on the ethnography done so far. We can set an agenda for the kinds of questions we want to ask – pushing people beyond a few geographical cases, for example. This will also help moderation questions be more directed.

Salient Themes and Finding Common Ground

As Alberto reported, core themes included the following (below). We focus here on themes which involved the most dynamic participant interaction (speakers referenced each other, voiced agreement/disagreement, mentioned repeatedly/stressed, asked directed questions to each other):

  1. Immunity certificates/passports are ineffective and risky, particularly as we do not know how immunity works in the context of this novel coronavirus.
  2. Contact tracing via Apps and other technologies represent a trade-off approach in which civil liberties may be sacrificed in the interest of fighting the virus. The risks here are three-fold:
    i. Data and privacy: we don’t have a clear privacy-friendly solution for the development of these technologies; it is also unclear where the data will be sourced from and how this will be regulated.
    ii. Unequal distribution of risk: there may be a disproportionate targeting of vulnerable communities,
    iii. Absence of the counterfactual: these apps may be a way of letting governments lift lockdowns (presumably over economic concerns), however, the effectiveness of allowing ‘normal life’ to resume is – as yet – untested: we are developing strategies without counterfactual knowledge to work with

Underlying these concerns is the question of necessity: why are these kinds of decisions being made? Why is there a push to develop track and trace technologies, especially if we lack evidence to support their effectiveness?

Participants conclude that no, contact tracing is not the answer to fighting the pandemic currently , but that they may allow us to prepare for the next pandemic or future waves of the current one.

  1. Coordination and Cooperation across governments and communities is more pressing than developing new technologies. However, the fear expressed by participants is that the pandemic has offered governments a pass to rapidly develop and implement surveillance technologies under the guise of public security and virus containment. It is agreed that we need to maximize access to participation in these efforts (e.g. through slow media approach, collaborative and constructive journalism platforms, information sharing, collaborating with researchers and experts and contact-sharing, as well as crowdsourcing existing data, tapping into open data sources. In this sense, focus should be placed on improving existing infrastructures, cooperating across governments and harnessing the potential of local communities.

Meta Themes:

  • Feasibility (in terms of resources, time management, coordination, monitoring): Do we have the capacity to process new evidence, the capacity to react quickly and to use the resources and legal systems available to develop policy?
  • Ensuring Security (on multiple levels): how do we avoid data being misused? How do we strike a balance between ensuring data privacy, anonymity and data security with containing the spread of the virus in a way that does not force a trade-off? How do we protect vulnerable communities? How does the rush to develop new technologies put medical professionals, their patients and communities at large at risk?
  • Transparency: Decisions we make about how to combat this pandemic need to be justifiable and their effectiveness and necessity need to be clearly definable.
  • Access: to information/reporting
  • Uncertainty: the absence of counterfactual examples/data makes it easy to speculate, but also makes any decisions we make riskier.
  • Temporality: Discussions in the call highlight an apparent clash of temporalities: what needs to be done in the HERE AND NOW to contain the spread of the virus? VS… What can we do to (a) prevent future pandemics and (b) be better prepared for them? These two aspects need to be untangled as they are currently being conflated. Furthermore, how and what can we learn from past pandemics, and what can this tell us about how we ought to approach the current as well as future pandemics?

Section 4: Moderation

We agree with Maria that we needed more moderators, at least one for voice and one for chat. We’ve provided some suggestions above in terms of question preparation. Main summary here: ask directed questions when needed, but don’t be afraid to let silence take over for 5 seconds. One of the things we are taught in ethnographic interviewing is that silence feels way longer to interviewers than interviewees. It allows people to collect their thoughts and think more deeply about the topic at hand. Changing the topic too quickly and artificially can lead you away from those deeper and connected insights. Make sure any directed questions build upon the topic currently being discussed.

We’ll have to brainstorm a way to link up chat questions to voice questions so as not to move too abruptly. We can also discuss the possibility of letting the chat unfold as it unfolds, naturally, as people may be more comfortable talking in chat form than speaking. As a result, be really careful with the direct callouts to individuals – it puts people who might not want to talk on the spot. Perhaps it would be best to only ask chat questions aloud if they are direct questions on the topic that is being discussed in voice – clarification questions, follow-up questions, etc. One of the great benefits of digital calls is that those who would not be comfortable speaking up in an in-person event might feel comfortable sharing their views textually – we have to be sure to leave room for this. In the same vein, ‘lurking’ isn’t necessarily a bad thing. It may lead to people posting on-platform later, if we create the right channels.

Moderation preparation:

  1. The host(s) develop a set of questions to ask based on the ethnographic data analysis outlined in section 1.1 and solicits input into those questions from the edgeryder team as well as on platform.
  2. In the call, these questions provide a guideline for the conversation but need not strictly be adhered to if the conversation veers from the topics identified (the whole point of this exercise to make room for unexpected insights to surface).
  3. Consider moderation best-practices and netiquette, and communicate these to participants beforehand. See for instance this recent ACM resource.
  4. Establish moderation netiquette with participants:
  5. Mute when you are not speaking.
  6. State your name often, especially when the call is without video.
  7. Do not interrupt.
  8. Have a good system in place for queuing questions or comments (via chat or by “raise hand-function depending on the software).
  9. Repeat research purpose of the call often.
  10. Mention and enforce the Code of Conduct.

Moderation execution:

  • Most calls will need at least two moderators, one leading the VOIP conversation and one monitoring the chat.
  • The chat moderator is responsible for documentation and information, i.e. asking people to mute, repeat their names (preferably by private DM), ensuring questions in the chat are relayed and capturing the chat after the call finishes.
  • Both moderators are responsible for dynamically keeping the conversation going, this means coaxing the conversation forward and allowing for silence when needed. Key here is ensuring the speakers have the space to speak and build on each other.
  • Moderation follow-up:
  • Send short “satisfaction” survey including questions like:
  • Did this conversation meet your expectations: Y/N + elaborate
  • Did you think the discussion was relevant to your life/work: Y/N + elaborate
  • Were there topics that were not discussed that you had expected to hear about
  • Follow conversation on the post about the event
5 Likes

I agree with that notion for interviews, that problem is that a call with many participants is also to a degree a presentation or performance and the silence sounds long for those listening. So we need to balance that carefully.

I asked most people via private message if they would like to speak before raising them up

We accidentally discovered another feature of the emergent SenseStack: if we do the call on now.edgeryders.eu, people access through their Edgeryders account. Then, it becomes easy to put through the consent funnel only the people who have not yet posted on the platform, and who have therefore not been through the funnel yet. Those who have posted before can simply be taken to the call’s room. Elegant! This is evolving more and more into a @hugi

But of course then we would need a recording function to work in now.

Other advantage: in data protection and data safety terms, the audio file would now live on our server, stewarded in the same conditions as the database of the main Discourse platform.

This is a good point. So far, I have taken the opposite path, and tended to start “on the dot” to signal respect for people’s time constraints etc. What do people think about it? @matthias? @MariaEuler? @nadia?

An important word is missing from the above. I would reformulate: a contact tracing app is not the answer to fighting the pandemic currently, etc. (see here).

How interesting! I had no idea.

I would add “use headsets, or at least headphones. And be in a quiet room” :slight_smile:

I like this idea a lot, though I expect low response rate.

2 Likes

That’s no contradiction with Alberto’s point of starting sharp to not waste people’s time. We can just set the appointment to start sharp (like 18:15) and give that time to everyone, but also mention in the description that people can join from 18:00 to have a virtual coffee chat, test their audio setup and so on.

3 Likes

That’s what we did this time around:

1 Like

could use c.t. and s.t.

These abbreviations are found at universities after times:

  • 10:00 s.t. means that the event will begin at exactly 10:00.
  • 10:00 c.t. means that the event will begin 15 minutes later, that is, at 10:15.

s.t. - Latin “sine tempore” = “without time”

c.t. - Latin “cum tempore” = “with time”
This is also referred to as the “Akademisches Viertel” (academic quarter-hour).

2 Likes

There’s only so much you can do in 2 hrs in a virtual room with many people who don’t know each other. Have you considered a series of calls- 2-3? If you limit the participation per call to 12 people or so, you can have more calls with more in depth treatment of the topic.
Also, can’t you do more focused one-on-one interviews after the calls, with selected participants?

If you think of online events as community events in general at Edgeryders, that means they would need similar amounts of preparations, what we call sustained engagement. We know with @nadia that a full engagement campaign takes months.
Examples of sustained conversations across different events:

  1. OpenCare workshops - each had a dedicated team with targeted engagement in a 2-3 months runup.
  2. The Reef-series of 3 workshops allowed for people to register over 2 months for basically what was one single topic.
  3. Coping with loss of work online series - now low key and has very little outreach done atm and still, the in-between online conversations can help fill in dots with socialization and followup to each call. Imagine what could happen with more communication support, more curation etc.

Otherwise, girls: thanks for a very high quality reflection <3 @CorinneCS @Leonie @amelia

3 Likes

Totally agree, @noemi. One of the things we were hoping to talk about was this time frame balance — we also tend to like months of advanced planning before events like this, to get the things we discuss above (diverse participants, depth of analysis). But at the same time, we of course understand the need/see the value in being able to do more of these events with less lead time. So we can work together to find what’s optimal/makes sense. The event was great and I think we can definitely build on it for future — even if we don’t get engagement on platform after, for example, the event itself was rich in material.

1 Like

I’m still not sure how effective one on one interviews are for the SSNA model, because they often don’t generate on-platform interaction. But in terms of deepening our own understanding, they’re of course great.

1 Like

Hm, I thought the interviews could be posted on the platform - like how it happened with the NGI Fellows last year.

Yes, definitely. I still have the same concerns re interaction, but I think that could be workshopped for sure.

We’ll have to brainstorm a way to link up chat questions to voice questions so as not to move too abruptly. We can also discuss the possibility of letting the chat unfold as it unfolds, naturally, as people may be more comfortable talking in chat form than speaking. As a result, be really careful with the direct callouts to individuals – it puts people who might not want to talk on the spot. Perhaps it would be best to only ask chat questions aloud if they are direct questions on the topic that is being discussed in voice – clarification questions, follow-up questions, etc. One of the great benefits of digital calls is that those who would not be comfortable speaking up in an in-person event might feel comfortable sharing their views textually – we have to be sure to leave room for this.

I would try to keep the voice conversation as one-pointed and moving naturally from topic to topic as I could while letting the chat be more free form. Chats always take on a life of their own anyway, and each person views it sort of differently. For some it is a way of saying something they regard as a more minor point. For others the way to introduce a link. For others a way to bring tangents into the picture without derailing the vocal conversation. And for still others a way to comment on how the voice action is going or to point out info about muting and audio quality. So I would accept that it’s going to be more diffuse.

Some will talk and some will only use the chat. I didn’t talk voice in the conversation because I thought it more important to hear the others. But I did make a couple of small points in the chat.

I think it is advisable to remind people to look at the chat even if they are otherwise 90% focused on the talk, because the moderator can send a private message without interrupting. But otherwise, the chat is optional - not the main event.

2 Likes

Maybe, we could try to personally invite and ask those participants to write a short post about their point, saying that we want to discuss their view further. let’s try

Yes, therefore I think it is best to have the conversations host focussing on making that flow and the chat host reaching out to the convo host if it seems important to include new points/questions/people from the chat.

1 Like

Yes I think that is a good approach.