Edgeryders tech strategy 2020 – 2024

Workspaces Software (General)
1

That’s a bit of a bombastic title for this post. What you’ll find here are my notes from the IT strategy discussion between @hugi, @owen and myself on 2019-11-20.

1. IT resourcing in general

  • In very general, we need to have more resources assigned to IT.

  • To justify that spending, we provide the strategic plan below.

2. Deployment infrastructure

  • We need a staging Discourse website.

  • We settle on Netlify for static website deployment for now. It’s a good option until we need sites with a backend / database. Can and should use a “netlify-production” branch on Git from which to deploy.

  • We will look at load times etc. to decide about using a CDN etc…

  • We will use Digital Ocean to place Virtual Machine images for experimental deployments. Images can also be stored there while not in use, causing less cost. This is a good option for any experiment that would “mess up” the production server by requiring major changes to its software.

3. Webkit for our project websites

Specialized websites for different events and topics is nice, but it’s too much effort for the outcome. Consumes a lot of time. They should be generated in a much more standardized way. That’s where Owen is going now. The first version of that is visible already on now.netlify.com.

The goal should be that even for a small project, somebody can put together a website based on components from an existing toolkit. Also the sites are similar to use for every new event.

Later, this kind of software would be used for “gradual introduction to Edgeryders”: they use that software first, the edgeryders.eu forum later. It could even be published right on the edgeryders.eu frontpage, moving the forum to forum.edgeryders.eu or similar.

Owen is developing this at the moment, based on ideas and input from @nadia. It will have several functions in its initial version, allowing to use it as a front-facing website for Edgeryders live events and projects:

  • A Discourse SSO login, using our communities.edgeryders.eu login provider. This already works.

  • A Twitter-wall like function, aggregating content from Telegram channel, a Twitter hashtag etc.

  • A way to select parts of this aggregated content as an admin and post it to Discourse as one combined post, using the Discourse quotation style that shows the Discourse profile image of the quote author alongside

  • A way of aggregating various media content by type from the Discourse website (videos, audio etc.), to make that content more visible.

Everything for that toolkit should be in Vue.js component form. That’s already done. James’ edgeryders-form software would also be integrated into this framework as a Vue.js component. To prevent a duplication of effort, it will replace the other form software component that Owen made once for a somewhat different use case.

Later, this framework would also include a login to edgeryders.eu, allowing to post to the forum based on activity in the front-facing website.

When this framework needs to talk to an API, we should try to not have a separate (caching) serverside component for this, but to always directly access the Discourse API. This way, we have one single API to access, and all the documentation, installation, access credential etc. issues have to only be solved once. Where the JavaScript web application would like to see a more comfortable / specialized API, there are two options:

  • Internal, client-side JavaScript library that provides this more comfortable interface. The server load of more requests sent to Discourse is not a major issue, as it’s just about short API queries with a JSON response.

  • Where necessary, we can extend the Discourse API with Ruby on Rails code, as we have done before.

4. Other strategic planning

We got four IT areas in Edgeryders: online conversation, events facilitation, collaborative sensemaking, internal processes. Collective intelligence and events software stuff is directly tied to income.

  • Internal standardization. Some decisions that we made during the 2019-11-20 meeting:

    • Vue.js as framework for JavaScript web application development.
    • Tailwind as CSS framework. Can be extended with additional components. 17k stars.

  • Open Ethnographer as a generic tool. Do we want to tie Open Ethnographer to Discourse as a general annotation solution? Hugi: Amelia says that our Open Ethnographer for Discourse is a “state of the art” solution for online ethnography already. So we could also go for a more general solution, like a browser plugin that allows to use Open Ethnographer on any website. This means, there are these two possible futures for Open Ethnographer. We can keep both open. At least at the level of data compatibility, even though it would probably be two parallel software projects that only share some components.

  • Mixed media. In the longer run, ideally we should mix media more. Like an app to podcast / videocast to the forum. That should ideally be in Discourse. Including features like “quoting audio”. That would be nice to post to Discourse upstream. Meant to be able to reach “unusual suspects”: people who are not great at writing. (by: Hugi)

    We could have a view of all content by Open Ethnographer, as text, audio and video, right on the public-facing portion of the Edgeryders website. In a way so that it presents a screencast, from which people can dive into the content and connect to people. That could be a new way of storytelling, and of generating income. (by: Hugi)

  • Upgrading Graphryder. It has to be upgraded. It should be integrate-able as a component into Owen’s frontend software. To show to users. We should leave Neo4j behind, as they provide crippleware with a lot of features only in the enterprise version. But there are multiple open source alternatives. Hugi: There’s all kinds of technical debt in Graphryder, even in the visualization part. Linkurious etc… So we might want to redo that part as well. Also including ways to visualize in a way that adapts more to the style we have in the frontend facing sites.

  • Moving from Riot to Discourse. There is currently an interesting feature convergence happening between productivity-focused business chat software (Slack, Rocket Chat, Riot) and modern forum software (Discourse). Both now include threads (in a flat threading model with reply-to information), subscriptions, some form of real-time notifications, @mentions, and even the same sets of emojis.

    So if we extend Discourse with (1) a real-time notification system that works well with mobile devices and (2) Slack-style dashboard of subscribed categories and topics (“channels” in Slack) and direct message topics (same in Slack), we won’t need Riot anymore. The quick-and-dirty discussions that should not enter the organizational memory which happen now on Riot should then simply be direct message topics in Discourse. (And we can even have a mechanism that erases these after some time.)

2 Likes
2

Nice wrap-up, and it sounds “grounded”.

One thing I missed (but maybe its already mentioned elsewhere) is a concise statement about Open Source / Free Software (e.g. preferred license, or a vague “where possible”).
I know this can sound annoying (sorry for me being so bitchy about it), but in my evaluation proprietary software poses a risk to all kinds of things that I would like to see in societies (to name just a view: innovation, alternative economic endeavors, freedom generally, control over critical infrastructure where and when its wanted, collaboration, transparency, equality, … and so much more). While this does not necessarily has to be included when talking about Tech Strategy it aligns with the Goals and where towards reaching these Goals tech could play a role. And still, a clear statement can have influence on the strategy.

Also there could be consideration about how to support alternative CDNs/hosters (e.g. CHATONS hosting collectives), even if they seem not yet ready for production from your point of view.
Do we want to see a more decentralized, open and fair landscape in the hosting business 2024? Like companies using ethical banks and stuff? Can we do anything about it? I sure think so.

2 Likes
3

Do you know of any credible/viable alternatives to github?

4

Thanks! This wasn’t in there only because it’s obvious to the people in the room (i.e. @matthias, @owen and myself). We’re even considering moving away from Neo4j for our graph stuff because of its semi-proprietary model.

On that note @matthias, I found RedisGraph. Graph database Redis module which uses the Cypher query language. It would be a good fit to build a graph database representation of the discourse conversational data in real time.

2 Likes
5

Amazing! If that’s mature enough, it’s the sweet spot to make both the graph nerds (=you) and software architecture purists (=me) happy :slight_smile:

The best possible outcome would be an extension of the Discourse API that makes all or nearly all Discourse content queryable as a graph database. Which is a generally useful thing for analysis, so it could become a well-loved Discourse plugin on which others can build other stuff …

1 Like
6

It depends on the “for what?”.

Seen as a code hosting platform it is basically serving as a git repository. As git is a decentral versioning software, I do not see much problems in also using github and the alternative could be just git itself.

Then, github comes with tons of pretty nice extra functionality (like issue tracking, project management, wikis, interaction with other services via hooks, analysis, Pull Request and Reviewe management, dynamic static website hosting, …).
And, because it is the de-facto platform, it can also serve as a “source code search engine” and has even some social features (like 'star’ing and following other users and repos).

Especially the later is not yet covered by any platform I know of. And here, it is difficult to catch up with github, just because of its sheer size and the network effect. We would either need a very strong platform to counter that or something like ActivityPub but for code and developers, so that a decentralized code(r) fediverse can emerge.

Most of the other functionality is handled by GitLab - at least in the Enterprise version I guess. Also gitea (a gogs fork) seems to cover the basics really well. Then there are project management tools with git integration. I specifically know redmine has (and do not judge redmine by its default looks), and I guess the more fancy ones like Taiga serve this as well. There must be plenty developer-oriented web frameworks for that (e.g. GNU Savannah).

Personally, I am not convinced by GitLab for my use cases. Gitea looks like it would hit the sweet spot for my use cases.

Then I have to admit github has some awesome features, besides it using Rails and letting Aaron Petterson (random talk) work on tough stuff for Ruby and I use it personally/semi-professionally too. But for a company it should be easy and cheap to setup e.g. gitea and just always also push to github automatically (for extended visibility) when pushing to the own ‘central’ git repository.

Does that somewhat answer the question?

2 Likes
7

Unfortunately it does, thank you :frowning:

1 Like
8

https://www.google.com/amp/s/www.theverge.com/platform/amp/2019/10/9/20906213/github-ice-microsoft-software-email-contract-immigration-nonprofit-donation

2 Likes
9

Maybe you want to open a new topic with your requirements/ideas regarding a github alternative? Maybe other folks have other alternatives in mind.

10

I’ve fleshed out a strategy for this here:

11

In a very interesting development, one of the people working on the tech for The Borderland just made this proposal completely independently of our developments:

I wanting to do a thing in Elm again and was thinking I could make an alternative frontend to Loomio. So we keep Talk as a CMS of sorts, but have a little bit more control of the presentation. Mostly it’s about removing clutter like the sidebar, loading screen, comments, threads list, etc.

This speaks for keeping the caching-microservice separate from Discourse, as it would allow building a frontend framework that could be used to present content from any public forum with an open API by just adding new calls to the microservice. Front end framework would always get identical data from the microservice. People who needed compatibility with Loomio, Vanilla or whatever they were using would write those functions and hopefully send us a pull request.

12

Mmh, no, I don’t like this kind of architecture :slightly_frowning_face: I think the era of “overly general software” is a matter of the past already – it was a failure, and Edgeryders was certainly a part of it with the Drupal “logical objects framework” type of thing.

Let’s keep close to the Discourse idea of “this is forum software, done well”. Discourse deserves a frontend site builder kit like we plan to create, but one that is done well: generic enough for other Discourse sites to use, but not for other whatever-websites to use. Given that, there is no need for a caching microservice layer. Discourse has caching inside already, using Redis …