review of your revised paper, “Semantic Social Networks: A Mixed Methods Approach to Digital Ethnography”, is complete. your paper is much improved, but there is still some work to be done. we enclose below comments from two reviewers to guide you making the final revisions.
as noted in our message of sept. 8, 2018, papers for FM must not exceed 6,000 words, including text, references, tables, figures, and abstract. allow about 100 words per inch for tables and graphs. the abstract must not exceed 150 words. you may choose to place one or more of your tables of figures in our online section for supplemental materials. those materials are supplied in camera-ready form but they must be truly supplementary – that is ancillary to the main arguments of the paper.
with regards,
H. Russell Bernard, Editor
Amber Wutich, Associate Editor
FIELD METHODS fmx.sagepub.com
reviewer 2
This (revised) paper proposes a mixed methods approach to analyse ethnographic online data by combining traditional qualitative data analysis and network analysis techniques. While the revised paper does a good job in responding to many of the reviewers’ comments and suggestions, I have some remaining concerns which would require a further revision.
Major points:
One of the main elements of the proposed approach is that it treats conversations in online domains as human communities. However, the term ‘human communities’ is not defined or explained and is used inconsistently throughout the text. For example, on p. 3 the term ‘virtual communities’ is used instead. A clarification of the term and how members construct a community in interaction would be essential since the analysis of the codes is based on members’ interactions. The literature on ‘online communities’ might be helpful in framing the discussion. => @alberto or rather @amelia?
While the coding process of the online data is much clearer in this version, the role of the researcher as participant-observer and coder (including biases inherent in this process) should be further discussed. To illustrate, on p. 7 you state “We visualize and treat ethnographic data in network form, simultaneously displaying the informants’ social network (who informants are talking to) and the content of their conversations (what informants are talking to each other about).”
I would argue that the content of the conversations as captured in the codes is an interpretation of what has been discussed by the informants, and not content as such. => @amelia
Visualizing the codes produced by the researcher in a network form is the main contribution of this paper. Yet the potential of semantic networks does not come as strong as it should be. I would encourage the authors to go beyond the technical and statistical parts of semantic networks (section 2.2) and discuss in-depth the possibilities of semantic social networks to discover meaning on different levels of analysis (i.e. collective beliefs and individual interactions). => @amelia
The social interactions among participants and interactions among codes are important components of the proposed approach. As stated on p. 7 (“Our chosen representation has only two types of nodes, participants in the conversation and ethnographic codes, and only three types of edges: comments (participant-to-participant), references (participant-to-code), and co-occurs-with (code-to-code).”). The ‘references’ and ‘co-occurs with’ interactions seem unclear to me. How do you capture the ‘participant-to-code’ interaction? As to the ‘co-occurs with’ interaction, I understand that this comes through the annotations, but what do you mean by content in the following sentence in which you describe that specific interaction “Its edges are induced between two codes that occur in annotations that refer to the same post or content” (p. 13)? Is the content referring to the whole dataset? @alberto
The description of the social network (section 4.1) is not very clear to me. More specifically, you end this part with the following sentence “These structural features allow us to conditionally accept that most opinions expressed in the forum have been run past someone (other than the proponent) in the conversation.” (p. 12).
What is the point here? Who is this someone else? And since that most opinions expressed in the forum run by him/her, would not be important to look into it? => @alberto
As the semantic social network is the main element of the proposed method, it would also make sense to present a network graph based on your analysis under section 4.2. We only get to see a part of this network by filtering edges by k>=6, and I am wondering why you haven’t included the whole network. => @melancon
reviewer 4
Overall, the paper is more comprehensive now as it covers a broader range of content. In doing so, it also becomes clear that the paper does not make a novel contribution in terms of methods. As referenced by the authors, the idea of coding for social actors and encoding their communication exchange has been proposed and demonstrated before. Also, a new method would need more stringent evaluation. The paper might be better positioned as an applied research study, where a substantive question would need to be addressed, e.g., related to the focus of the online community that was studied. => @amelia (in the letter alone).
(I took the liberty of moving this to a new topic.)
I understand what reviewer 2 wants, it does not look like much work. But what should be done with reviewer 4? She is basically saying the paper should not be published, or not on Field Methods.
I reckon we respond to reviewer 2 with edits (because some edits to 2 will more clearly articulate why our method is worthwhile) and then just write a couple sentences to 4 saying in brief why it’s a new method (aka, we respectfully disagree). Worst case and they ultimately reject, we take our paper elsewhere.
Seems like 1 and 3 were satisfied, so if we satisfy 2 that’s 3v1 anyway.
I tentatively assigned the remaining observations across authors – please see the edited text above . This is a much simpler rewriting work than the one we did in 2018; hopefully we can get it through in days, rather than weeks. I also updated the Overleaf:
One of the main elements of the proposed approach is that it treats conversations in online domains as human communities. However, the term ‘human communities’ is not defined or explained and is used inconsistently throughout the text. For example, on p. 3 the term ‘virtual communities’ is used instead. A clarification of the term and how members construct a community in interaction would be essential since the analysis of the codes is based on members’ interactions. The literature on ‘online communities’ might be helpful in framing the discussion.
I was going to write “we know since Rheingold (2000) that `virtual´ communities are simply real human communities that communicate through computer networks” and leave it at that, but maybe the circumstances demand an answer more rooted in anthropology. So, maybe another one for you, @amelia?
This leaves two items for me, which I already addressed both in the paper and in the letter. Now you…
Sure, let’s use that Rheingold sentence you wrote, then standardize language to ‘online communities’ (or ‘virtual communities’) and cite a few people. Can use anthropologists like Boellstorff and Miller. TBH I think this comment is a little silly, everyone is pretty clear on what this means by now, but am fine to be more meticulous if they want. I’ll update.
Do we need a call or can we do all of this asynchronously?
I go through the issues assigned to me (done), updating both the paper and the letter.
You do the same
@melancon or @jason_vallet make a nice visualization of the larger graph. We may not use it, I think it will be your typical hairball unfit for visual analysis, but in this case we will put it in the letter.
We re-read and shorten as needed. In my part I did not really lengthen anything, it was more re-phrasing. Perhaps I added 10 words or so.
We already know the visualization is pointless, this is exactly why we use the edge filtering trick. Are you suggesting we make the picture anyway to display it in our response letter?
Hmm, I just re-read the whole section “Semantic social networks”. It may be that it is badly written, but we do make a case that our approach differs from those proposed by others. One could argue that our approach is an extension on Renoust’s – we simply do not cite Renoust’s paper! How can I agree with reviewer 4 saying “As referenced by the authors, the idea of coding for social actors and encoding their communication exchange has been proposed and demonstrated before.” We argue the exact opposite.
Would that picture including all tags be ok (to back up our response to reviewer 2’s quoted comment)? Colors are from Louvain (ending into 10 different non trivial classes and a dozen trivial ones, isolated nodes).
I guess so. A “hairball” picture, accompanied by two sentences on why we think it is not processable by looking at it. Ideally, the famous Melançon paper about undecypherability of dense graph would also be invoked.
