As we migrated data to our new platform, we realised that most records in the ethical consent funnel database table showed as null. It turns out a bug in the Drupal module let users created after the funnel deployment date (2016-07-08) bypass the funnel; it stores the account creation date as the default value of the consent validation date. This did not show up in testing because, of course, we tested with accounts created before that date. A second bug (but this time in Drupal core) seems to have overwritten the consent validation date for some of the user accounts created before the deployment date.
Proposed solution
We ported the consent funnel to the new platform (fixing the bug!), and in fact extended it to all Edgeryders, not just to opencare. We tested carefully (it works) and keep monitoring it. New participants or those returning to the opencare discussion after porting to the new platform will have to go through the funnel, and so solve the problem for themselves. On Monday August 14th (in the first three weeks after deployment), 68 users had completed the consent funnel.
After the October festival, we will generate a list of participants that have not gone through the consent funnel. We will contact them one by one, and give them a link whereby they can go through the funnel and register their consent, even without a need to post further. @noemi will be responsible for this action.
Marco: I guess this needs to go through our ethical advisors. Can you point them to this post? I am of course available for any further questions.
Acknowledged. Just wanting to clarify, correct me if I’m wrong:
the funnel now pops up for everyone when registering on the site. We are looking to make sure it was completed only for users who posted/ commented in Topics tagged project-opencare?
@matthias if you give me some instructions on where to look I can put together that list myself - I will anyway need it.
I am bringing this up to the advisors this week, at our call. My two pennies is that if the people who posted without completing the funnel could be identified, they should be proactively informed, and asked if they confirm their consent… Actually, their reactions could constitute material of research in its own right, if a group of those who correctly passed through the funnel at the time were to be contacted as well.
Hardly ever researchers go back to discuss the consent, and how the understanding of the participants evolved over time…
This could be a tough nut to crack… the questionnaire to collect info would have most likely to be thought from scratch, and those validate on an independent sample… and the final cardinality of the questionnaire should be commensurate to the sample size we have available…
Since this is a rather peculiar situation (we have a sample, we do not plan a new recruitment), can we share at the order of magnitude of the people involved in this situation, so that we can reverse engineer the cardinality and the requirements for validation?
~300 Participants in OpenCare on the Drupal
– 50 (?) those with a valid consent funnel registration date on Drupal.
– 80 (?) those that, at the end of October 2017, will have gone back to the new platform and completed the consent funnel here.
The precise list of contributors with missing consent has been produced, and an agreement on the process to acquire the consent has been agreed. It is based on one-on-one email. Noemi is executing.
Of course, we do not expect a 100% success rate. Our position is: the consent funnel is an extra precaution, but – given Edgeryders’ collective publishing platform nature – it does not alter the nature of the pact between users and platform. Recall also that the license on Edgeryders content never changed. Ping @markomanka and, through him, the ethical reviewers.
Circa l’azione: nel frattempo abbiamo stimato in modo più esatto il numero dei consensi mancanti. Si tratta di 192 persone, con una stima “larga”. Larga vuol dire questo: alcune persone hanno partecipato alle discussioni di OpenCare, ma i loro contributi non sono stati inclusi nella “coda” di contenuto per il coding etnografico. In termini tecnici, i primi sono reperibili analizzando il contenuto di /tags/project-opencare, i secondi quello di /tags/ethno-opencare. Di fatto, stiamo chiedendo il consenso anche a persone che sono intervenute solo per fare una domanda, o per organizzare il servizio cucina all’Open Village Festival.
Di queste 190, 105 ci hanno già dato il consenso. Si vanno a sommare alle circa 160 che ce l’avevano già dato. Il lavoro continua, ma naturalmente a rendimenti decrescenti. Speriamo di ottenere ancora 20-30 consensi, realisticamente sarà difficile ottenerne di più.
Sarebbe bellissimo potere studiare gli “abusivi”… ma di fatto quelli che troviamo si trasformano in non abusivi, e quelli che non troviamo sono irreperibili.
Ottimo il suggerimento sull’analisi delle persone che decidono di non dare il consenso. @matthias, is there a way we can see how many people, upon being confronted with the consent funnel, decide to drop out? Google Analytics returns the bounce rate for each page, but I am not sure that our consent funnel is a page, with its own URL that we can look up on GA.
There’s nothing for this right now. (The consent funnel is shown as a client-side JavaScript popup when a user clicks + New Topic, so it will be a portion of the logged-in users dropping out from category pages and the category overview frontpage.)
A reasonable solution to add this functionality would be to let the consent funnel JavaScript do its own API calls to Google Analytics upon being shown, and upon being accepted. If that is desired, please post an issue on Github. If we can do without it, let’s not do unnecessary work though …
Caro Alberto e tutti: avete fatto veramente il massimo, e questo è proprio il comportamento eticamente (e scientificamente) corretto. Avremmo un web migliore se tutti avessero la vostra attenzione e la vostra “cura” per gli utenti.
Avanti tutta
Wow Antonio, grazie per il tuo tempo e atenzione… Dopo di avere reazioni misti (?) al nostro email per prendere gli consenti qui mancavano, mi trovo un puo piu ottimista leggendo le tue parole.
192 consent-less accounts (of which one is a system account, created by @matthias to perform technical tasks, so 191)
110 give consent
71 No answers after 3 attempts at over 1 week long intervals
5 deny consent
6 NA - email addresses not found
We will now update the relevant field in the edgeryders.eu database. This will take care of the 110 consenting individuals for future research as well as for opencare.
Next, we will prepare an update of the harvesting script, used both to create the Graphryder database and to export opencare data for release. We can do this via a list of “special people” that have not consented to being part of the study. The content associated to these people will not be harvested. User IDs will be hashed in the data export, as agreed.
Now, the important question is: how do we build the list of special people? Re-reading the comments by @Mario.Falchi and @Antonio it seems to me that:
Mario recommends that the list of special people includes the 5 people who do not give their consent + the 71 who did not reply + the 6 who were unavailable.
Antonio thinks that, once we have made our best effort to acquire someone’s consent, we are more or less OK using that person’s content. There is a forgivable problem of form, but not one of substance. It follows we should only put the 5 dissenters on the list of special people.
First of all, congratulations for dealing with this so swiftly @noemi !
My call would be closer to what you identify as @Mario.Falchi’s, but I see his and @Antonio’s as emphasising two sides of the same coin.
Legally this is but a problem of form, which is resolved once the concerned people are able to express their opinions.
However, we should deal with the 71 people who did not reply under the precautionary principle. Let’s exclude them from opencare and further analyses, and obfuscate/erase their posts.
Concerning the 6 who did not share even an email address: had they actually posted? Should I understand this was an anonymous form of posting in which no online search could trace the nickname on other platforms, and no personal information was disclosed? If so, although I would exclude their postings from further analyses (but this is more of a personal call than anything, honestly), I would leave them readable online.
Now to the important part. Did the 5 who refused to consent share any comments with you about this decision of theirs? This is where we have real insights into our communication, and their perception of risks… and how the funnel affects users’ behaviours. - In pragmatic terms, we have to exclude them from opencare and further analyses, and to obfuscate/erase their past posts… but their replies to you could be excellent learning material for internal reflection. Not for research, since they did not consent ultimately, but for internal reflection, since if they so did, they offered an explanation spontaneously.
Concerning the 110 consenting late, it is them that I would place under special observation, to observe if anything in their pattern of communication/posting changed between before and after going through a consent.
Understood. But, sorry, we do NOT obfuscate or erase their posts. Edgeryders’s TOS are very clear as for that. What we do is not use their content for this research.
The 6 unreachable ones are most likely people who changed their email address in between. These are not people that were involved with opencare in the last two years; rather, people who had in previous years posted content that we found consistent with the opencare research questions. Edgeryders admits the use of one-time email addresses, but no one uses them that I know of. And, if we want to be fastidious about it, all posting in ER is anonymous, as we do not enforce, nor even recommend, a real name policy.
As for the question about why five people denied consent, I will leave the answer to @noemi.
