RyderEx Installation Manual

RyderEx builds and displays network graphs that describe conversations on Discourse platforms annotated with OpenEthnographer. This manual documents how to set up the RyderEx stack from scratch.

RyderEx architecture

  • RyderEx import script exports data from a Discourse & OpenEthnographer postgresql database and builds a Neo4j graph database from that data.
  • RyderEx Neo4j database is the data layer of RyderEx.
  • RyderEx GraphQL API and dashboard are deployed together in a Docker container and are how the end user interacts with the network graphs.

Preparing a fresh install of the RyderEx stack

RyderEx has only been tested on Ubuntu 20.04.

Neo4j 4.3+ and APOC

See official installation instructions.

After installing, enable and start the system service.

$ sudo systemctl enable neo4j
$ sudo systemctl start neo4j
$ sudo systemctl status neo4j

Then, run cypher-shell and set a password for the neo4j database.

$ cypher-shell

You must also install the appropriate version of APOC for your Neo4j instance. See the official APOC instructions.

After installing APOC, you need to add two line to the APOC config file.

$ sudo nano /etc/neo4j/apoc.conf

Add the following lines

apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=false

Then restart the neo4j service

$ sudo systemctl restart neo4j.service

Python 3.8.6+

In case you need other version of Python on your system, you may consider managing them with pyenv.

There are two required Python libraries.

$ pip install neo4j psycopg2-binary

Install the import script

$ git clone https://github.com/edgeryders/ryderex-import-script.git
$ cd ryderex-import-script
$ cp config.example.json config.json

Edit the config.json file with the right authentication information for your neo4j database and the Discourse postgresql databases you will be importing from. You can import from multiple Discourse instances at once by adding multiple database configurations to the ‘databases’ array.

Run the import script

You should now be ready to build your Neo4j database for RyderEx. The import script does the following:

  • Gets all users, groups, topics, posts, categories, tags, annotator codes and annotations from Discourse.
  • Redacts everything that is not public.
  • Dumps this cleaned data into batched JSON files that are optimized for fast loading into Neo4j.
  • Imports the data into Neo4j and creates the graph relationships.

Run the import script:

$ cd /opt/ryderex-import-script
$ python graphryder-import-psql.py

On the main server, there is a crontab job that runs the following scrip every night:

#!/bin/bash
. /home/webmaster/.bashrc
export PATH=/home/webmaster/.pyenv/shims:~/.pyenv/bin:"$PATH"
pyenv local 3.9.1
cd /home/webmaster/ryderex-import-script
python graphryder-import-psql.py

Install Ryderex

To install RyderEx API and dashboard, follow the instructions in the RyderEx repository.