I am returning home from the kickoff meeting on a new project. Something interesting happened there: for reasons that I won’t go into right now, the partners find it inappropriate to do our online collaboration on proprietary tools and cloud services owned by stacks (GAFAM). This turned out to be practically impossible. We do have repositories for papers and deliverables (zenodo.org); we do have the Edgeryders platform for conversation. Other things turned out to be hard.
Collaborative document editing. There is still little in the way of replacements to Google Docs. Some people in the group had tried Cryptpad, but not liked it. Elsewhere, @ton suggested Collabora. Several alternatives were proposed, from Overleaf (will we be writing our notes in LaTEX?) to versioned Libre Office Documents circulated over email. I would propose there ARE some alternatives, as long as you are prepared to go minimal over features (which is good for you anyway). For example, Etherpad has become quite beatiful, a bit like the Hackpad of old. Some disadvantages, of course: it saves in SQLite, so even if you download a backup you won’t be able to do anything with it. It is part of Sandstorm, which is self-hostable (and includes an app for Jupyter Notebook, which looks real-time collaborative too).
Cloud archival. We are talking here of humble documents. Certainly not papers, not even project deliverables (those go on Zenodo). But the minutes from the meeting, the working document, the unfinished draft, the picture. Most orgs use Google Drive, or Dropbox, or Office in the Cloud. Again, not so many alternatives here (Ton likes Nextcloud) In the end, we solved this one with… FTP .
Event registration. Do we even need a non-evil Eventbrite alternative? It is not clear to me. Among other things, those data are problematic from a GDPR point of view. My suggestion of going back to the Barcamp tradition (run events through a wiki, people “register” by writing their name into a wikified list of attendees) was not met by much enthusiasm.
So, here is where we are. If you want to go non-evil in our everyday workflow, you need to make quite some efforts, and it is not even clear you can find a proper solution. This, even standing on the Egderyders small, but solid tech infrastructure built by the vision of @matthias and @daniel. What a mess!
Some more options for synchronous collaboration tools to discover and choose from:
For spreadsheets, LibreOffice Calc has synchronous collaborative editing (see).
Using desktop software for editing complex documents is preferable anyway, as recreating that software inside a web browser is a dirty hack from a software architecture perspective: complex software in interpreted JavaScript, what to expect if not a performance nightmare and a memory hog?
For documents, GitHub’s programming environment Atom together with Teletype enabled synchronous collaborative editing of plain-text documents. We’d have to write in Markdown and find a way to get a live preview in a second column, like here in Discourse. Markdown is quite ok even for writing long and complex documents (example on this platform) and has the advantage that people cannot create a mess using incoherent formatting. Finalizing the look & feel could be done by one person after import to LibreOffice.
For databases, Obvibase is pretty interesting. We used it for the Future Makers Nepal project, instead of that ugly hack of using online spreadsheets (like Google Sheets) as an ad-hoc database. It’s not open source software, but run by one friendly programmer guy.
For raster graphics drawing, DrawPile is an amazing open source cross-platform application that allows synchronous collaboration, including pressure sensitivity that can be used with graphics tablets or certain on-screen styluses (ThinkPad X series Tablets, Apple pen etc.).
For quick collaborative drawing sessions e.g. to illustrate concepts during a conference call, several collaborative sketching tools are available: sketch.io Sketchpad, iscribble.net, twiddla.com and others.
For vector oriented drawing that allows synchronous collaboration, so far I don’t know of anything that could replace Google Draw.
For time tracking with multiple users, it does not look bright. The best cross-platform tool that I could find that comes with local data storage and an optional self-hostable server is Watson, with the server-side companion Crick. This is however a command-line tool. The situation is so dire that my brother started an open source project recently called tom, with the user interface in tom-ui. Will be ready to use in 1-2 months.
And for everything else, the Frama services list is quite interesting. These are 33 free software tools that you can use in the version hosted by Frama or download and install yourself. Most are well-known open source applications like Etherpad Lite that were re-branded by Frama for their services. Which is a bit annoying, but anyway, it’s useful.
For file hosting plus collaborative editing of text documents, spreadsheets and presentations, the following solution is (as far as I can see) the only available full-featured and self-hosted option so far. In addition, it is open source.
A version of LibreOffice Online. As listed on that page and explained here, there are several options:
Vanilla LibreOffice online, which lacks integration with a file storage solution as such.
Collabora Online, which is open source software with commercial support (i.e. you pay for support to get access to the code, but the code itself is open source). There is an online demo.
CODE, the Collabora Online Developer Edition. It seems to be the exact same codebase as Collabora Online (see), just that it “will warn you when it starts to look like it is being used in an enterprise – ie. more than twenty documents or ten users” (source). Of course, since it’s open source software, that can be disabled.
A self-hosted file storage software with LibreOffice Online integration. About 20 options are listed here, though for our purposes Nextcloud seems best, followed by ownCloud.
Optionally, other self-hosted open source software that integrates with LibreOffice Online and the file storage solution and allows us to replace Google Calendar, GMail as a web-based e-mail client, and other tools.
Out of the box, Nextcloud already comes with the following extra functionality: chat and video conferencing (incl. mobileapps), CalDAV backend to sync between compatible calendar applications, CardDAV backend to sync contacts between compatible applications, and a “Federated Cloud ID” system that allows to share files and folders with anyone signed up on other instances of Nextcloud, ownCloud or Pydio (I suppose so that they see the files in their own instance).
Notably, there is Kopano. It also integrates chat and video conference calls.
Affordances
What we get from this combination:
Fully self-hosted and “stackless” solution.
Synchronous collaborative editing of text documents, spreadsheets and presentations. The way this works in LibreOffice Online is just like in Google Docs etc.: you see the changes and selection made by others in realtime. There is also “File → Save”, but that just means everyone’s changes live in memory on the server until somebody saves them (to Nextcloud, for example).
In-browser editing that works on mobile devices, as demanded above. For that, all text selections come with large handles that can be moved on a touch based device.
You can download the actual original document format, unlike in Google Docs / Sheets / Presentations where you can only download exported documents (which may look different when re-importing to the Google application).
Since Nextcloud files can be opened via file syncing and WebDAV mounts on a local computer, editing them with the desktop LibreOffice applications is totally possible. To avoid version chaos, we would need a file locking function or convention so that only one person can check out a file for local editing.
Advantages over Office365 and Google Docs / Sheets / Presentations due to the unusual, image tile based rendering of documents in LibreOffice Online (see this conference talk for details):
No memory or performance issues with large documents, unlike Google Docs. The browser only renders a few images, no actual text. This creates a small lag when updating content, but not more than when editing Google Docs content in a non-trivial document. Certainly ok to work with.
All document look exactly the same in the browser interface, mobile browser interface, desktop application (LibreOffice) and in print.
The user interface is essentially the same as desktop LibreOffice, including advanced features like a style editor etc. that are not available in Google Docs for example. Complex dialogs like the style editor are rendered as “canvas” elements, which means they are images in your browser, rendered on the server, complete with the Windows UI look & feel. But speed-wise, these dialogs are ok to work with, so there’s nothing much against them.
LibreOffice Online will not be a buggy nightmare. While there are still a few bugs to rule out, they are all related to the communication with the server – and that is comparatively simple, with the server sending only images. The server itself is stable software, running “standard LibreOffice” with an API interface.
Options for extension and further developments: since all of this is built around LibreOffice and its Open Document Format, there is quite some future potential for own and others’ developments, including the following (that I would personally like to see):
Making LibreOffice online respect the LibreOffice desktop application’s lock file mechanism. That way, while one user checks out a document for editing in a desktop application, others can only view it but not edit it, whether using LibreOffice Online or LibreOffice. Or in the case of Calc, multiple users can edit using the desktop application.
Support for collaborative synchronous editing in LibreOffice Calc (which already works) in combination with others editing the same document in LibreOffice Online.
Synchronous collaborative editing also in the LibreOffice Writer, Draw and Impress desktop applications, all in combination with others editing the same document in LibreOffice online.
Looks good, as far as I understand (which is not much: “tile rendering”?). My main objection to Libre Office is that it has too many bells and whistles. I like Markdown because it nudges people towards simple, clean files. But maybe we can have an iron convention that all headings have to be done in Heading 1, 2 and 3 only, and other than that only lists, bold and italics are allowed.
Tiles are small images of (here) 256×256 px that fit together like tiles on a floor. Whenever you change a letter in the document, the change request is sent to the LibreOffice Online server. The server updates the affected tile(s) according to the change: the letter, reflowing words into lines as needed after inserting the letter, etc… Then these tiles (images) are sent to your browser and shown there.
So your browser never shows the actual letters you type, but an image of the letters as rendered by the server. Unlike Google Docs, Office365 etc… It’s quite a funny choice of architecture, but it works, and even offers some advantages over Google Docs etc… It’s comparable to how OpenStreetMap (and Google Maps in its early days) renders its maps in the browser.
Yes, something like an iron convention. It is quite simple to enforce with LibreOffice styles, which are available as paragraph styles, character styles, and page styles: we provide a template with styles, and the convention is that hardcoded formatting beyond the styles is forbidden. All hardcoded formatting can be removed easily with Ctrl + A and “Format → Clear direct formatting” (also works in LibreOffice Online).
Actually I love LibreOffice templates. (So much that in 2006 I won their template competition with this.)
When going through my list of business ideas, I just came across a name for a business that would offer “going stackless” as a commercial service, migrating a company out of Google, Amazon etc. to self-hosted solutions. (And I think that since Snowden, or at latest since 2018, this is a pretty good business idea … .)
And the name I had in mind was: ANTIFOG Cloud Removal Service
Of course, Facebook is also one of those corporate tools that everyone wants to get rid of. Here’s a proposal how that could be achieved, and I think it could be worth a try during our next two big projects in the next few years.
It starts from the observation that, for the intellectually minded people in the Edgeryders crowd, Facebook still serves these functions:
A medium of contact with friends and acquaintances. That’s covered by Facebook Messenger mostly, and we don’t aim to replace that. Messenger is still bad, but at least separate enough from the rest of Facebook that we can get rid of the rest. (People who want to get rid of the Facebook website can empty their feed there by unfollowing every single person while keeping the account, and then keep using messenger.com and / or the Messenger app.)
A curated feed of articles and news. Newspapers had this function of curation before, until Facebook’s feed algorithm made everyone take on that role for their friends (without payment). And worse, a lot of newspapers now serve either bad content (clickbait) instead of journalism or they hide behind a paywall, or even do both (like many German ones). That means newspapers cannot serve an interesting curated selection of information to intellectually minded people, who will often rather rely on an eclectic mix of articles from around the web, curated by their Facebook friends, or maybe by their sub-reddit.
That second function is what we could replace with the Edgeryders platform, and as an effect increase the amount of time people spend around here by a large amount. It could work as follows:
We would employ 1-2 content curators (let’s say, from Nepal ), potentially paid with project money for community management or engagement. They take 2-4 hours each per day to curate a feed of content in a dedicated category here on Edgeryders. Since Edgeryders is (in a weird way) a homogenous community, the same kind of feed will be interesting for everyone here. It has to be a lot, though. Probably ≥100 links per day, so people can easily spend an hour per day to go through the articles they find interesting from it. For the presentation format, we’d chose something like that of our current company blog. So each link would have a proper preview in the feed, and users also have the option to visit the Discourse topic in order to comment / discuss with others about that link.
The content curators will need some external input, of course. But if 10 here on Edgeryders would support the scheme by forwarding interesting links and tips to the curators, we could get it started. And finally make Edgeryders a place to hang out as well, not just to work …
I know that @johncoate likes to post interesting links to Campfire quite often. This would be similar, just more of that so that it can feed people their daily dose of interesting information …
Right - I only do it if I think it directly pertains to something we are working on or might work on. And I only see a fraction of what is out there.
And Facebook is never going to be even remotely socially responsible and I think they will continue to mislead their users about what they are really up to. Also, whereas Google employees protest and even cause change in some ways (less sexist bro culture domination for one thing), Facebook employees all seem to have drunk some sort of kool aid that makes them seem more like members of a money cult. Talk about people who eat their own dog food…
The added value would be “time spent on our platform”, no? And while they are here, people would voluntarily contribute content that is relevant for our community.
In other words, our online community is not yet self-sustaining / stable. Whenever we stop with (funded) engagement and outreach efforts, very few people post anything here. How can we change that?
Tool collections
I observed quite some smaller organisations and networks collecting tool collections in 2018. Listing all the self-hostable solutions is not feasible if not done proper itself (collaborative, versioned, and “exact” - which is difficult - you can sell and use libreoffice as a book-keeping application if done right).
Will get a bit technical now:
Nextcloud
As an nextcloud and CODE/Collabora user I can say that I am pretty happy with it. Also, e.g. the austrian habitat uses it with kind of SSO (Single Sign On) and central user authorization, with integrated discourse and dokuwiki. The setup and deployment is not real fun yet, though. Btw latest nextcloud allows definitions of templates (regarding the discussion on iron conventions), but these pop up only if the web-editor is used of course
The nextcloud WebDAV does work in principle, but is not really feasible if you have a small or un-tuned server, big amounts of files and users or old Linux clients (until Ubuntu 16.04 I stumbled upon some serious bugs and sometimes horrible performance in the evaluated implementations). I would assume that the LibreOffice file locking mechanism works fine with WebDAV if the remote “directory” is mounted. But LibreOffice comes with a webdav-implementation itself - if I am not mistaken - so it probably uses different access and locking mechanisms depending how it identifies the underlying filesystem (which could be a remote webdav).
(online!) Time-Tracking
Depends on whether you want time tracking or project management with timetracking. Personally, I use redmine for the later but with its bureaucratic overhead its not a solution that will work for everybody. In the community we use the anuko time tracker, which is probably “as modern as redmine” (both tool really do not have the trello feel), but does its job really well. But I understand the need for an offline-also solution.
framasoft
Because framasoft was mentioned I wanted to point to fairapps/fairkom, they are as dedicated to free-as-in-free-speech software as framasoft, offer nextcloud and other tool hosting and offer professional service.
That’s valuable input. I think we’ll definitely give CODE / Collabora a serious try.
Oh, that’s a good idea. I didn’t really think about it yet before, but now that we have set up Discourse SSO anyway for our Discourse hosting offer we could take it a step further and make our other tools provide authentication via the Discourse SSO provider as well.
I’ll also have a look at the Anuko time tracker – thank you for the hint! If it provides an “auditable log of actions” on the time records, it would be great for our time tracking needs for the Horizon 2020 research projects.
I was just made aware by @alessandro_mambelli of the series of articles “Goodbye Big Five”. It’s both hilarious, inspiring, and at times cringeworthy (from a tech perspective). Thank you Alessandro, and you might find some good tips / content above
ONLYOFFICE. An open source office suite that allows collaborative editing using browser-based editors. Also comes with other aplications for e-mail, CRM etc… See also in Wikipedia. It works with HTML5 canvas elements, so similar to Collabora but not the same – canvas based rendering could be faster. It can integrate with document storage in OwnCloud, Nextcloud etc… Also, mobile applications are available (but these are not open source).
Canva for Work. A browser based, collaborative drawing tool quite similar to Google Draw. Not free though, but stackless.
I evaluated OnlyOffice a tiny bit, but no “native” support for Open Document Format (all documents will internally converted to the Microsoft format of which I willingly forgot the name, but this thing like xlsx), and a generally more closed surface than Collabora made me decide against it.
For diagramming, there is the really neat draw.io. Can be integrated into nextcloud. No real-time collaboration yet - you have to be a bit careful not to open the same file with two users at the same time (that means, if shared, share read-only or include a warning box or something like that into the diagram for proper workflow).
.
