I’ve been thinking a lot about how to manage large-scale ethnographic coding, particularly in different languages. From thinking through my experiences with coding + analysis in Open Care and working with Nermine on Open Village, I want to propose we push further into the open notebook science terrain that we have been discussing since the beginning of Open Care. @Alberto and I had a discussion this morning to explore the idea further.
I think that the key to keeping the coding standardised, accessible, and transparent is to have an open codebook that all ethnographic coders commit to updating systematically and regularly. This creates 1) a way of ensuring rigorous coding and 2) keeps ethnographers working on the project accountable to one another.
Traditionally, all ethnographers coding their data keep a codebook. But this codebook is usually kept private, often on pen and paper, and doesn’t get published alongside the final product. Asking an anthropologist for her field notes or codebook, in usual disciplinary practice, would be strange. Further, as ethnography is usually a solo pursuit, the ethnographer only has to make the codebook make sense to herself. Instead, I propose a codebook that remains open throughout the coding process, attentively updated by the ethnographic coders and an object around which frequent discussions of coding choices can occur.
So what is a codebook? It is a living document that helps the ethnographer document and keep track of her coding decisions.
For each code, the following should be included in the codebook :
- brief description - the name of the code itself
- detailed description - a 1- 3 sentence description of the code’s qualities or properties
- inclusion criteria - conditions that merit the code
- exclusion criteria - exceptions or particular instances that do not merit the code
- typical exemplars - a few examples of data that best represent the code
- atypical exemplars - extreme or special examples of data that still represent the code
- “close, but no” - data examples that could mistakenly be assigned this particular code
- similar codes - if this code doesn’t fit, what other codes might be useful instead?
(5 and 6 are useful but since OE can auto-list examples of a code, may not be entirely necessary if we need to keep it more simple. At least 5 may not be.)
For our project, we can add:
- This code in other languages (linked)
Codebooks also include memos, which are brief descriptions capturing the ethnographer’s thought process in using the code. As Glaser (1978: 83) puts it:
‘(A memo is) the theorizing write-up of ideas about codes and their relationships as they strike the analyst while coding… it can be a sentence, a paragraph or a few pages… it exhausts the analyst’s momentary ideation based on data with perhaps a little conceptual elaboration’
Memos help the ethnographer keep track of her thoughts during the coding process. They are immensely useful in later stages of qualitative analysis when trying to theorize about the data, like when we will be writing up reports at the conclusion of the project. They help the ethnographer:
- remember connections she made off the cuff,
- formulate alternative hypotheses to those she made before,
- propose new codes or question existing ones (especially in cases when not confident enough to want to make a change then and there),
- integrate thoughts notated in previous memos or field notes,
- help formulate or articulate concepts that are not yet fully-formed enough in her mind
- help her future self or other researchers make sense of her choices.
The last point is crucial for our purposes — keeping these kinds of memos in the codebook will help immensely when trying to undertake collective ethnographic coding.
Examples of memos:
memo 1: used ship speed
since Luke was talking about outrunning other ship in Millennium Falcon
memo 2 : changed ship speed
to insufficient ship speed
to be more precise since there were negative complaints about the Millennium Falcon’s speed.
memo 3: I think it’s possible that Han is trying to establish dominance through sarcasm and bravado, but is actually intimidated by Luke’s undetermined powers. Luke seems oblivious to this dynamic but concerned with the group’s survival.
The Open Ethnographer course goes into a lot more detail on ethnographic coding, so do let me know if this doesn’t make enough sense and I can add information to explain further!
Looking forward to continuing the experiment