How to hard-update a Graphryder dashboard (obsolete)

Workspaces Software (SSNA)
1

This thread refers to a version of Graphryder we no longer use

There is a trigger to update the Graphryder API. By “update” we mean here re-import the data from Discourse, and place it in its own Neo4J DB.

It works like this:

1. Make sure the relevant content is tagged for harvesting

Graphryder collects data on the basis of a (Discourse) tag identifying the project. By convention, these tags always start in ethno-. For NGI Forward, for example, it is ethno-ngi-forward.

To make sure you have the right collection of topics, you need to tag each topic with the project’s tag. Most projects in Edgeryders have their own cats: for NGI Forward, the cat is Internet of Humans. Occasionally community managers and ethnographer assign an ethno- tag to some content that lives elsewhere (for example in past projects), but normally every top in the project cat is part of that project’s ethnographic corpus. This means, every top in the project cat needs to be tagged with the project’s tag as well.

To do this:

  • Go through all public sub-categories plus the /none sub cat (example https://edgeryders.eu/c/ioh/none.
  • For each of these sub-cats, scroll down until the end of its topic lists
  • Next, go back to the top and click on the Topics button to the left:
    image
  • Click on the Select all button. A wrench icon appears on the right. Click it.
  • Click Append tag, then select or type the ethno- tag as appropriate.
    Select all topics and append the tag ethno-ngi-forward

2. Re-import your data into Graphryder’s database.

@hugi says:

Just visit:
https://graphryder1.edgeryders.eu/api/hardUpdateFromEdgeRydersDiscourse

UPDATE: each graphryder dashboard has its own URL, with the format https://graphryder.edgeryders.eu/PROJECTSHORTNAME. The second part of the URL above should be added to that. For example

https://graphryder.edgeryders.eu/ngi/api/hardUpdateFromEdgeRydersDiscourse

Never do this just before a demo though as it takes about 30 minutes to rebuild.

3. Regenerate the graph

After it’s done (you’ll see the numbers tick up as you refresh the dashboard), go to the dashboard, open the right hand menu and click all the graph regeneration toggles.

2

The hard update does not work reliably. Right now I am seeing a 502 error. Full traceback in the file attached.

error_502.txt (5.1 KB)

3

It looks like Neo4j is choking. Maybe it’s running out of memory, I know that neo is pretty memory intense. @matthias?

4

Other times I get this:

image

5

… however, if I relaod the dashboard, I see the user and post counts keep going up… fingers crossed.

6

There’s 8 GiB of free main memory around now. It did run with 1-2 GiB free memory in the past, so I doubt we got a memory issue. But you can check of course, as you have the SSH login now.

1 Like
7

Just writing to ask what the status is on the Graphryder dashboard updates/errors? Both projects are due for a viz and writeup so just checking in on when we think we’ll have it working. Thank you! @alberto @matthias

8

I think we have all the pieces together now for me to repair the POPREBEL and NGI Forward installations, since Hugi fixed some issues on another installation before going on vacation. I’ll attempt the repair today and report back …

9

I fixed the Graphryder dashboard installations. See:

Looking at individual posts from within Graphryder still does not work (that’s an old issue). Everything else seems to work again. cc @alberto

2 Likes
10

Thank you so much. Very helpful. I can already see more meat than the last time it worked…

11

Thank you!! Perfect timing for our meeting today :slight_smile: I will also let everyone know about the OE priority list and being as parsimonious as possible with requests.

1 Like