I’m posting our (@amelia @CCS @Leonie) Methods Reflections below:
Ethnography Team: Reflection on a Surveillance Pandemic
Section 1. Setting up the event in future
- Identify a topic for the event based upon the ethnographic analysis done so far. Ask: what topics are the community most interested in? Which would be most useful/generative to do a deep dive into?
- Have at least a month to plan the call, invite a diverse range of participants (including the most active community members on platform on this topic), and put informed consent procedures in place.
- Use the SSNA/codes/ethnography to pull out questions to ask participants during the call. (Research team can be present and ask questions as well as moderators).
- Record the call and transcribe the recording.
- Post the transcription and code the transcription.
- Generate an SSNA based upon it (we need to brainstorm ways to have the Social part of the SSNA visualised in this, but at worst it’s a visualisation of the main themes of the conversation. At best, we separate each contribution into a different comment to see what concepts most frequently co-occur together).
Section 2: Ethical consent funnel and procedures
Create an information sheet to put on the event page that is very easy to locate and is in clear, human language. This should also be present before the Zoom link is reachable/above the Zoom link. This needs to be done at least a week before the event, but preferably as soon as possible. This information sheet needs to contain at a minimum the following:
- Description of Research Project (what the data will be used for).
- Description of Event - duration, type of event, what they can expect to encounter.
- That the event is being recorded.
- How the event data will be made public, processed, and stored, and for how long (include the possibility that the call will be transcribed and posted).
- That they have the right and ability to withdraw at any point (up until the publication of final research results, give date).
- Who they can contact if they want any of their contributions removed/want to withdraw.
- Potential risks and benefits of participation (in this section, we must list that we cannot guarantee them anonymity or data privacy, in part because we are using an insecure video client)
- What body is overseeing the research: if ERC/H2020, for example, then mention that an ethics committee with ethics advisors is overseeing the project, and give their contact information.
- That they also agree to abide by these rules in terms of data protection and circulation – they are not permitted to make information public beyond what the research project allows in point 4.
Get participants to digitally sign an informed consent form with the above information at some point before they enter the call.
- My thoughts on best practice for this: create an online consent form through something like Qualtrics or Google Forms (not sure about the security of the latter, would have to look into it-- or generate own internal form if we use the ER platform to video call). When the user submits the form, they are given the link to the video call and can enter. That way you know no one in the call has not gone through the consent funnel.
- Example of an info sheet/consent form currently being used: This area is password protected [401]
- Noemi’s “Meeting Registration” page would indeed work, but only if the I’m not a robot box was changed to a “I consent” box.
- At the beginning of the call, remind people that the call is being recorded and of the information on the information sheet (particularly on their ability to withdraw/opt-out).
- Inform all participants when the data is posted on the platform. Underneath/on the same post as the recording, give contact information for who to contact if they want to change their consent status and reiterate that they can do so.
- Host the call on a secure platform so we can assure participants of a higher degree of data privacy/lower risk of data breach.
We also need to be clear about what the participants are allowed to use the data for. It was clear that there were a few journalists on the call looking for a scoop/for information to publish. We need to be clear that part of the agreement is that you as a participant can’t go off and write using any kind of identifiable data.
Section 3: Content and demographics:
A. Content is shaped by WHO is participating: notes on participant demographics and its implications for content
-
Networks and recruiting participants:
The virtual meeting consisted of 36 participants including experts within the legal, medical and public health, digital tech, privacy and human rights, public policy and media communities. Participants in the call were invited through existing social and professional networks, i.e. were largely existing contacts of the Edgeryders team, primarily Alberto (friends-of-friends approach). Thus, many were based in Italy, some in the UK, Sweden, Belgium and France. Expanding and diversifying our recruitment for future events will allow us to hear from a broader range of experts, researchers and practitioners. We should brainstorm ways to do this effectively, perhaps with @noemi and @nadia (and others experienced in outreach and engagement). -
Encouraging Participation:
That being said, a full demographic overview is limited by the small number of introductions given by the participants. We should think of ways to give speakers the opportunity to introduce themselves (if they would like to/are able to) – in a manner that is comfortable to them. The chat was one useful way of doing this, but it’s not clear that chat introductions lead to more engagement between people of similar interests, since they came quickly and densely at the beginning.
In virtual events lately, @amelia has been doing a virtual version of the 15 minutes for coffee built in to schedules before conferences formally commence. One way of getting people more comfortable and chatting might be to schedule 15 minutes before the call as a “coffee and chat” period, a soft entry. This happened point to a certain extent, and could be further encouraged by making it an actual thing on the schedule.
This kind of space for introductions (or thinking of other ways to do intros) may also encourage participants to become acquainted with one another before the discussion begins, and might foster more interaction across a larger number of participants during the call. Of the 36 participants, we find a small handful of experts on call participating more than once/speaking more than once. The conversation was largely driven by 4-5 main voices + mediated by community managers.
The same goes for the conversation in the thread following Alberto’s report on the meeting. How do we maximize participation and encourage participants in calls to engage on the platform after the call? Also a topic we can brainstorm together.
- Making Discussions more Representative:
The following report is based on the exchange between a small number of speakers (in the discussion and chat), and though there are some salient themes, this may not be representative of the concerns/ideas of the broader community.
To that end, the participants (or at least those who were the most active) represented ca 3-4 European countries (incl. the UK). How do we include a broader range of people in these discussions? In terms of empirical examples, we mainly focussed our discussion on the UK and Italy, two of the hardest-hit countries currently affected by the virus. We need to bring in cases from countries with different experiences/ cases where it’s ‘going well’: e.g. Germany, Finland, the Czech Republic etc.
This could be helped in our next call by pulling out questions we want to ask beforehand based on the ethnography done so far. We can set an agenda for the kinds of questions we want to ask – pushing people beyond a few geographical cases, for example. This will also help moderation questions be more directed.
Salient Themes and Finding Common Ground
As Alberto reported, core themes included the following (below). We focus here on themes which involved the most dynamic participant interaction (speakers referenced each other, voiced agreement/disagreement, mentioned repeatedly/stressed, asked directed questions to each other):
- Immunity certificates/passports are ineffective and risky, particularly as we do not know how immunity works in the context of this novel coronavirus.
- Contact tracing via Apps and other technologies represent a trade-off approach in which civil liberties may be sacrificed in the interest of fighting the virus. The risks here are three-fold:
i. Data and privacy: we don’t have a clear privacy-friendly solution for the development of these technologies; it is also unclear where the data will be sourced from and how this will be regulated.
ii. Unequal distribution of risk: there may be a disproportionate targeting of vulnerable communities,
iii. Absence of the counterfactual: these apps may be a way of letting governments lift lockdowns (presumably over economic concerns), however, the effectiveness of allowing ‘normal life’ to resume is – as yet – untested: we are developing strategies without counterfactual knowledge to work with
Underlying these concerns is the question of necessity: why are these kinds of decisions being made? Why is there a push to develop track and trace technologies, especially if we lack evidence to support their effectiveness?
Participants conclude that no, contact tracing is not the answer to fighting the pandemic currently , but that they may allow us to prepare for the next pandemic or future waves of the current one.
- Coordination and Cooperation across governments and communities is more pressing than developing new technologies. However, the fear expressed by participants is that the pandemic has offered governments a pass to rapidly develop and implement surveillance technologies under the guise of public security and virus containment. It is agreed that we need to maximize access to participation in these efforts (e.g. through slow media approach, collaborative and constructive journalism platforms, information sharing, collaborating with researchers and experts and contact-sharing, as well as crowdsourcing existing data, tapping into open data sources. In this sense, focus should be placed on improving existing infrastructures, cooperating across governments and harnessing the potential of local communities.
Meta Themes:
- Feasibility (in terms of resources, time management, coordination, monitoring): Do we have the capacity to process new evidence, the capacity to react quickly and to use the resources and legal systems available to develop policy?
- Ensuring Security (on multiple levels): how do we avoid data being misused? How do we strike a balance between ensuring data privacy, anonymity and data security with containing the spread of the virus in a way that does not force a trade-off? How do we protect vulnerable communities? How does the rush to develop new technologies put medical professionals, their patients and communities at large at risk?
- Transparency: Decisions we make about how to combat this pandemic need to be justifiable and their effectiveness and necessity need to be clearly definable.
- Access: to information/reporting
- Uncertainty: the absence of counterfactual examples/data makes it easy to speculate, but also makes any decisions we make riskier.
- Temporality: Discussions in the call highlight an apparent clash of temporalities: what needs to be done in the HERE AND NOW to contain the spread of the virus? VS… What can we do to (a) prevent future pandemics and (b) be better prepared for them? These two aspects need to be untangled as they are currently being conflated. Furthermore, how and what can we learn from past pandemics, and what can this tell us about how we ought to approach the current as well as future pandemics?
Section 4: Moderation
We agree with Maria that we needed more moderators, at least one for voice and one for chat. We’ve provided some suggestions above in terms of question preparation. Main summary here: ask directed questions when needed, but don’t be afraid to let silence take over for 5 seconds. One of the things we are taught in ethnographic interviewing is that silence feels way longer to interviewers than interviewees. It allows people to collect their thoughts and think more deeply about the topic at hand. Changing the topic too quickly and artificially can lead you away from those deeper and connected insights. Make sure any directed questions build upon the topic currently being discussed.
We’ll have to brainstorm a way to link up chat questions to voice questions so as not to move too abruptly. We can also discuss the possibility of letting the chat unfold as it unfolds, naturally, as people may be more comfortable talking in chat form than speaking. As a result, be really careful with the direct callouts to individuals – it puts people who might not want to talk on the spot. Perhaps it would be best to only ask chat questions aloud if they are direct questions on the topic that is being discussed in voice – clarification questions, follow-up questions, etc. One of the great benefits of digital calls is that those who would not be comfortable speaking up in an in-person event might feel comfortable sharing their views textually – we have to be sure to leave room for this. In the same vein, ‘lurking’ isn’t necessarily a bad thing. It may lead to people posting on-platform later, if we create the right channels.
Moderation preparation:
- The host(s) develop a set of questions to ask based on the ethnographic data analysis outlined in section 1.1 and solicits input into those questions from the edgeryder team as well as on platform.
- In the call, these questions provide a guideline for the conversation but need not strictly be adhered to if the conversation veers from the topics identified (the whole point of this exercise to make room for unexpected insights to surface).
- Consider moderation best-practices and netiquette, and communicate these to participants beforehand. See for instance this recent ACM resource.
- Establish moderation netiquette with participants:
- Mute when you are not speaking.
- State your name often, especially when the call is without video.
- Do not interrupt.
- Have a good system in place for queuing questions or comments (via chat or by “raise hand-function depending on the software).
- Repeat research purpose of the call often.
- Mention and enforce the Code of Conduct.
Moderation execution:
- Most calls will need at least two moderators, one leading the VOIP conversation and one monitoring the chat.
- The chat moderator is responsible for documentation and information, i.e. asking people to mute, repeat their names (preferably by private DM), ensuring questions in the chat are relayed and capturing the chat after the call finishes.
- Both moderators are responsible for dynamically keeping the conversation going, this means coaxing the conversation forward and allowing for silence when needed. Key here is ensuring the speakers have the space to speak and build on each other.
- Moderation follow-up:
- Send short “satisfaction” survey including questions like:
- Did this conversation meet your expectations: Y/N + elaborate
- Did you think the discussion was relevant to your life/work: Y/N + elaborate
- Were there topics that were not discussed that you had expected to hear about
- Follow conversation on the post about the event
