Hosting unMonastery online Services

Workspaces unMonastery
, ,
1


Together with @almereyda we would like to follow-up on conversation about hosting unMonastery online sites & services. We could coordinate it with edgeryders hosting setup as well as various hosting needs of other projects incubating here, but not necessarily. As unMonastery we have heavily overdue card for organizing dedicated VPS and Jon already came to the rescue couple of times migrating our matera.unmonastery.org site and few other services to infrastructure he manages. We re-started conversation about it on Ecobytes + unMonastery = :heart:?

@Ben @Kei @Matthias @chrono @guaka

2 Likes
2

Edgeryders hosting documentation?

@Matthias, looking at https://edgeryders.eu/en/edgeryders-dev wikis, I didn’t notice one describing current hosting setup. Do you have it documented somewhere?

3

Should be no problem

I can’t find server documentation either, but I recall that early this year we had already offered Edgeryders projects (with reasonably light traffic) free hosting. In fact, I run my own blog on the same server as edgeryders.eu! We run our own virtual fully FLOSS machine with some dedicated cores. If need be, and within reason, we can also beef it up.

4

i like sandboxing

I like using multiple virtual machines for sandboxing, this way different people can have root access to those VMs without need to worry about messing up setup done by someone else. As I said we could take a look at current setup of Edgeryders hosting, just to see if it makes sense to somehow coordinate it in near future. At the same time I would like to first focus on appropriate setup for sites/services running on *.unmonastery.org and *.matera.io

5

Hmm, what does coordinating VPS hosting mean?

As Alberto said, edgeryders.eu and some smaller supporting sites (Piwik, books.edgeryders.eu etc.) run on one Host Europe VPS. 8 GiB RAM, 16 GiB dynamically, two dedicated cores, four dynamically. Ubuntu 14.04 LTS plus ISPConfig 3 as control panel for shared hosting. Host’s nothing too powerful, but works with some optimization. (Like, remember to use PHP-FPM on VPSs, as I recently found out …). I have isolated sites against each other (permissions-wise, plus Jailkit), so there’s nothing against taking on fairly standard and not-too-heavyweight other sites like Alberto’s blog.

You seem to have other needs though, as you mention getting a dedicated VPS to have own full control without potentially messing up others’ sites. I don’t see an option to coordinate infrastructure among multiple VPS systems. Except if somebody from the “Edgespace” wants to set up their own ethical hosting company and physically host the VPS host system. (That will certainly not be me, as I kind of, em, despise the VPS superstructure as unnecessary complexity for self-hosting. If somebody like Host Europe manages that for me and does it well, I’m ok with VPS from an interface point of view, but if I ever set up servers physically it will always be one machine, one host.)

What was your specific idea about coordinating hosting infrastructure around VPS?

6

no need to optimize prematurely!

Since we discuss with @almereyda and Ecobytes our hosting needs. I just thought to do it out in open and communicate with you in case your would find interest in changing your current setup, or know about some other hosting needs among people participating in Edgeryders. Once we set things up properly we will document it and share here for the record and as invitation to possibly cooperate further.

I really wonder how exactly you manage backups, monitoring, logs, notifications (SMS?). Also if you happen to get stuck somewhere offline for 2-3 days (I believe you will not get hit by any bus with your great karma!), I guess @Alberto and/or someone else have all the means to manage various possible challenges which might come up in that time?

@chrono, @almereyda, @guaka and possibly few other people participating in ER community, have tons of experience in this field and possibly could offer their expertise in reviewing your current setup, suggesting optimizations, and maybe even sharing responsibilities of sysadmin :slight_smile:

7

Can you explain?

Sorry, @elf_Pavlik, I have lost you. What does this mean?

  1. You are still discussing the hosting need of the unMonastery, and no decision has been made yet.
  2. The unMon's hosting needs are too sophisticated for being compatible with the setup offered by Edgeryders.
  3. The unMon's hosting needs are compatible with the setup offered by Edgeryders, provided some changes/optimizations are made, and you are asking Matthias to consider them.
8

2

based on how i understood setup offered by Edgeryders from Matthias’ comments

+ getting drifted in conversations here quite a bit :slight_smile:

i find more and more appreciation to constrains put by platforms like github, which don’t enable sub-threads

9

sysadmins, step forward :slight_smile:

But sure, if somebody wants to share sysadmin tasks, welcome to step forward. There’s indeed a need for hands-on contributions (not though for proposing optimizations etc., since we’re nowhere near the level where I don’t know the next steps to improve the setup already …).

So yes, there are many ways to improve the setup, and to lower maintenance efforts. What we have though is a basic configuration that works and keeps the data safe (not caring for the last tiny error message, 101% uptime and all that … it’s called the 80/20 rule). As for backups, we have a week of daily full server snapshots on an external system, plus some months of optimized database backups, with daily success notifications sent to me. @Arthur and @Alberto have all the necessary server credentials just in case.

10

a page with documentation?

I really appreciate you taking your time to explain it to me here. At the same time I think compiling a document which presents it nicely would provide quick (just drop a link) reference in a future and avoid such useful information getting drowned in the see of comments. While I would find it a useful resource in general, personally I don’t really need it urgently so I do NOT ask about prioritizing it.

11

> edgeryders.eu and some

> edgeryders.eu and some smaller supporting sites (Piwik, books.edgeryders.eu etc.) run on one Host Europe VPS. 8 GiB RAM, 16 GiB dynamically, two dedicated cores, four dynamically. Ubuntu 14.04 LTS plus ISPConfig 3

This is pretty much the same setup as id22:'s current server, just w/ CentOS. I’m using some modifications, though:

* nginx as reverse proxy in front of apache GitHub - Rackster/ispconfig3-nginx-reverse-proxy: [Discontinued] An Nginx reverse proxy plugin for the ISPConfig3 control panel.

  • no webalizer for certain hosts (doesn’t work) Issues · ISPConfig / ISPConfig 3 · GitLab

  • ssh user root always goes to ~/private Issues · ISPConfig / ISPConfig 3 · GitLab

  • usually custom error messages disabled

  • Ajenti as additional server management panel (doesn’t interfere) for reverse proxy etc.

  • sudo npm install -g pm2 to make it available to all users, then create a website + ssh in ispconfig, deactivate it and create the reverse proxy rule in ajenti. could be automized by improving ispconfig3-nginx-reverse-proxy

> I don’t see an option to coordinate infrastructure among multipleVPS systems. Except if somebody from the “Edgespace” wants to set up their own ethical hosting company and physically host the VPS host system.

This is what Ecobytes is actually doing. Coordinating VPS infrastructure. Btw., ISPConfig also offers integrations for virtualization systems and/or multiple servers. Never tried it, but just in case.

> how exactly you manage backups, monitoring, logs, notifications

For monitoring, I like to use uptimerobot.com and would love to see something similiar from FLOSS world.  what did you mean by notifications (SMS)? That warnings get sent by SMS or anything more? @pavlik-elf

I believe the ELK stack (elasticsearch-logstash-kibana) would be nice for greater deployments. why didn’t you like elasticsearch again? because its java @chrono ?

> > As for backups, we have a week of daily full server snapshots on an external system, plus some months of optimized database backups, with daily success notifications

These are the regular ones from HostEurope or a custom solution?

> But sure, if somebody wants to share sysadmin tasks, welcome to step forward. There’s indeed a need for hands-on contributions

Let’s talk @Matthias

>

  • You are still discussing the hosting need of the unMonastery, and no decision has been made yet.
  • The unMon's hosting needs are too sophisticated for being compatible with the setup offered by Edgeryders.
  • The unMon's hosting needs are compatible with the setup offered by Edgeryders, provided some changes/optimizations are made, and you are asking Matthias to consider them.

I was a bit deranged by these three possibilities offered by @Alberto , but yes, initially we wanted to secure unMon’s hosting situation with @Kei and @Ben .

But the actual hosting needs are compatible, as I’m using a similiar setup where the stuff runs now. What we tried to propose was a seperate machine so non-PHP software doesn’t get in the way of PHP stuff. Which makes handling a lot more easy, in the end. If you’ve ever used uberspace.de, you knew how well shared hosting can work if you have the right tools at hands.

The general question underlying was how we produce infrastructure that is adaptable to flexible needs, looking at microservice architectures and any non-PHP software that nowadays way too quickly moves to AWS, Heroku and the likes.

In the end, if we IndieHosters joined forces, we could easy provide an OpenStack environment that suits multiple needs. But the participating organizations would have to aggree on this goal.

12

monitoring / logs

one alternative to kibana

is grafana + influxdb based on graphite

13

Matt is the man

On this, Edgeryders will stand by whatever Matthias decides to do.

1 Like
14

Amen!

I said at least 3 times that I just want to discuss it here out in open and simply invite Edgeryders to engage without any expectations. Please also note that this post here serves only as satellite/bridge for where I hope we can have actual conversations about unMonastery setup with everyone interested in this topic Root server sharing under supervision of Ecobytes? - The Federation - Allmende

Myself I also stay fully open to current setup of matera.unmonastery.org and *.matera.io simply going down or someone else stepping up to take leadership on taking care of it! @Kei @Ben

1 Like
15

“But the actual hosting needs are compatible, as I’m using a similiar setup where the stuff runs now. What we tried to propose was a separate machine so non-PHP software doesn’t get in the way of PHP stuff.”

Not really my view of things … I see VPS mostly as a tool to not get in the way of other people, and vice versa. Having to ask a server admin for installation of anything out-of-the-ordinary is frustrating for both the admin and the site operator, and so is the other alternative of doing major tasks on a host that you don’t fully know. So I suggest that the total friction (technological plus social combined) is lower when unMonastery and Edgeryders LbG (the company) each have their own VPS or root server. The unMon crew may let us know if this is not playing out budget-wise :slight_smile:

And yes, the unMon crew taking a VPS slice at Ecobytes sounds like a good “local hosting” option to me. Edgeryders LbG has no current need to switch to a different VPS, but once there is, we might also want to consider the Ecobytes service.

“The general question underlying was how we produce infrastructure that is adaptable to flexible needs […]. In the end, if we IndieHosters joined forces, we could easy provide an OpenStack environment that suits multiple needs. But the participating organizations would have to aggree on this goal.”

Yes, now on to the bigger picture! So let me be a bit crazy / visionary. I don’t see the IndieHosters competing with Hetzner, Host Europe etc. in the future: if somebody has money to spend on hosting at all, these bigger corporations have the economy of scale in their favor. However, IndieHosters (nice word btw) would be much better for bulletproof hosting, and for hosting for people with no money for it, including those many from the so-called developing world who will come online soon and bring their bright ideas. Read, hosting for barter value :slight_smile: Of course this requires running the service (nearly) without putting in money oneself: fast software on used hardware, powered by photovoltaics, in the basements of well-connected people … . I like the idea because it connects to my local supply vision but can’t start such a project for simple capacity reasons. If someone starts it, I’ll gladly help prototype with edgeryders.eu as a guinea pig :slight_smile: This is obviously for the medium term though.

“These are the regular ones from HostEurope or a custom solution?”

The database backups with notifications are a Drupal-specific custom solution. The rest is Host Europe. Except for long-term backups that I forgot to mention, which are also custom.

About hands-on help with server administration, maybe we can start a small collaboration around ISPConfig, since we both use it? There are some rough edges that maybe we can find / develop solutions for together …

16

They will still need some cash

(After meeting up with Ecobyte folks at 31C3). @Matthias, this is a nice vision, but indiehosters will still need (some) cash. We could support their work by being a paying customer: we are already paying for hosting, and could decide to pay them instead of Host Europe. This would give them some headroom to engage in building digital commons, no?

1 Like
17

Ecobytes and IndieHosters

We discuss working with Ecobytes (non-profit) on hosting infrastructure. You had chance to meet Gualter during his workshop on Hacking capitalism: IT organisations, structures and services to support the societal transformations.

IndieHosters (also non-profit) offers hosting on few selected services, all based on Libre Software and #IndieWeb style of decentralized networking. Not sure if you had chance to speak with its co-founder Michiel de Jong who also visited 31C3 Edgeryders Assembly couple of times. I really like that they encourage people to start more independent ‘IndieHosters’ like them and offer to similar service. So at some point Ecobytes could also spin another IndieHoster provider.

HTH

1 Like
18

Sure thing

I used indiehosters (small i) as a generic name for ISPs like you describe: FLOSS + decentralization. Could be Ecobytes or Indiehosters (capital I), or any other that people like, I am not competent to judge: just making a general point about ER being at home with that kind of provider.

1 Like
19

makes sense

I find term indie, quite strongly associated with IndieWebCamp, hence IndieHosters

I must admit also not seeing some non ambiguous term for such non-profit, community based providers of hosting services…

20

Let’s support unMonastery to get an Ecobytes VPS first

Yes sure, valid point about the cash needs. As I argued above, I propose that Edgeryders LbG and unMonastery each maintain their own VPS host. unMonastery is seeking for such infrastructure atm, so I propose Edgeryders LbG helps them top up what they can spend on this, and then they go ahead and get an Ecobytes VPS.

I also like the idea of hosting edgeryders.eu on an Ecobytes VPS and being their paying customer, but I hate migrating servers since it’s always some days of work and people get on your back if you don’t minimize downtimes with all means possible. Which is why I suggested that Edgeryders LbG only switches servers once our setup becomes messed up enough to deserve a major overhaul anyway. Could be two years still.

2 Likes