Assignment: Getting Realities to MVP

#1

Realities is one of the projects we’re bringing into Particip.io. It’s a tool to make visible the actual dependencies and leaderships in a decentralised organisation, and the idea behind it is described in this white paper. To give some context, this tool is being developed with a certain organisation (The Borderland) in mind as to not make the concepts too abstract, but its features will not be specific to that organisation. From the mission statement summary in the white paper:

We will develop a software system that helps people in a decentralised organisation plan and execute co-created events without traditional top-down management. As to not make the use case too abstract, we will use the organisation The Borderland and its participants as our user group and develop explicitly for them, while not adding features that are only relevant to that organisation. Realities should give users the ability to create Needs and then Responsibilities and to assign a Guide and a Realiser to each Need and Responsibility. Needs and Responsibilities are connected by Dependencies which clarify how they are connected. Creating and assigning Needs and Responsibilities and connecting them with Dependencies should be simple and intuitive. The UI of the software should also transparently indicate the correlations of these elements to each other for a faster overview of the larger scope, for example by drawing graphs and connecting them to each other.

Development of Realities started in December of 2017, but halted after a few months as priorities shifted. We got quite close to an MVP stage though, so our first priority is to finish that work and get the codebase to a stage where Realities can be deployed and used. We also want the codebase to be well documented and structured so that it can be worked on collaboratively in an organised manner.

In light of this Erik Frisk, one of the developers who laid the ground work for Realities in 2017, has been allotted two weeks from the freelance developer budget to do the following:

  1. Refactor and document the code to prepare for collaborative development
  2. Implement removal of Needs and Responsibilities
  3. Adding Guides and Realizers
  4. Adding dependencies between elements

We think that two weeks of work should be enough, but will discuss extending it if necessary.

Erik, feel free to introduce yourself and post about major milestones. :slight_smile:

#2

Just to say @hugi that i’m very much up for some beta testing on this service once things are into a more developed stage. I couldn’t help you code it if my life depended on it, but i’m keen to test the application of the software. One of the organisations i’m attached to, (that long term i’m hoping will be a potential to integrate with Edgeryders/OpenVillage ideas) is becoming increasingly decentralised and non-hierarchical [mostly through design - partly through lack of funding to employ senior level staff] and any tool that can help us streamline communications and task management would be a real boon to us.

#3

Duly noted! We will definitely make use of that opportunity.

#4

Just jumping in to say I will be watching closely. Really interesting, you two.

#5

Same for me.

#6

Hiya! I’m the guy working on Realities to get it to MVP. I helped write the first code a few months ago and am excited to continue in this forum :smile:

I’ll post updates here as I go. I worked on authentication in the platform a couple of weeks ago. After a few days break to work on another project I’m back and working on refactoring the frontend in preparation for adding the rest of the features we need for an MVP. I’ll be working part-time on the project for the next few weeks.

Looking forward to seeing Realities come to life!

#7

Quick status update :slight_smile:

I’m chugging along and working a few hours on the platform most days. Refactoring the frontend is going really well and it’ll be easy to add the features we need to take the platform to MVP once I’m done with that milestone (which is soon).

One notable change I’m making: During the initial hackathon we set a direction for the frontend where we’re using inline editing rather than an edit view to update needs and responsibilities. I’m swapping that around so that we use an edit view instead. There’s two reasons for this:

  1. The library we’re using for inline forms, riek, has 300 stars on github and hasn’t been updated for a year. If we switch to an edit view pattern we can use something more conventional like Formik, which has 10,000 stars on github and is continuously maintained.
  2. I think an edit view will actually be easier for the user to understand and cause fewer accidental edits due to missclicks, etc. This is up for debate however and I might be wrong about this.

It might be less exciting but I think it’ll save us (and users) some frustration down the line.

As soon as I have a working refactor (more like refactor + upgrade) I’ll push to github so everyone can see.

#8

Another update :smile:

The only thing I have left to do for the UI refactor milestone is to re-implement “creating needs and responsibilities” using Apollo’s new Mutation component.

One thing I wanted to mention here (if only to be able to link back to it later): Before I began refactoring we have several ways of styling elements in the code. We had agreed on using styled components, but in practice we had:

  1. Styled components, with styles defined in the component files where they were used (correct)
  2. Bootstrap CSS, applied globally (fine)
  3. A few custom css styles that were applied globally (not ok)
  4. A few styled components that were defined in a “styles” js file and imported in the components where they were actually used (not ok).

I’ve refactored so that we only have concepts 1 and 2 in our codebase. Feel free to argue if you don’t agree :wink: It’s ok to define styled components outside of the component where they are used if they are clearly reusable, but if so they should be thought of as their own presentational components and put in the ui/src/components/ directory individually (styled components are just normal React components after all).

Since the refactor is taking a while I’ve pushed the branch I’m working on in case anyone is curious and wants to take a look: ui-apollo-refactor

:heart::heart::heart:

#9

I’m finally done with the frontend refactor/update and can now continue with implementing the last few features of the MVP. It’s all pushed to master and live at http://realities.theborderland.se/

In my last update in this thread I said that the only thing I had left to do was re-implement “creating needs and responsibilities”, but I had forgotten about the search feature. That took some time. A big part of the refactor was making sure that we use Apollo to fetch data from the backend in a sane way, rather than fetch everything from the database when the page loads and then refetch everything every time the user triggers a mutation. The previous search was built on the idea that we have the entire database cached in the frontend. Now it actually fetches results from the backend based on the search term entered by the user.

The things I have left to do are:

  1. Deleting needs and responsibilities
  2. Updating reality guide and realizer for needs and responsibilities
  3. Adding and removing dependencies

I’m running out of time on my 80 hour estimate, but I think I can get all this done now that the code is in a better state. :heart:

#10

The last couple of days I’ve worked on “soft deleting” (setting a “deleted” property on) needs and responsibilities. It turned out to be tricky since the neo4j-graphql-js library doesn’t have a system for filtering nodes that are fetched using their @relation graphql directive. I managed to get it working in the end but it drained a few hours. It’s pushed to master and deployed if you want to take a look at how I did it.

@hugi: Deleting nodes and responsibilities is merged to master, but without all the proper authentication. Right now anyone can do anything in the system as long as they’re logged in as a user. I only have 6h25m left on the budgeted 80 hours. What should I prioritise? Some options:

  1. I can work on editing reality guide, realizer and dependencies as per the original plan, but I don’t think I’ll have time to implement all the proper authentication then. This is probably the best option for the unveiling on October 20.
  2. I can work on getting all the proper authentication in place for the features we have now.
  3. I can write better documentation for the stuff I’ve done so far, mainly focusing on guidelines for developing the frontend.

If you think it’s a good idea I could also keep working on the project for a while longer if there’s room to expand the budget by a few hours.

What do you think?

#11

Points 1 and 3 are priority. We need point 1 to reach the MVP goal for the demo on the 20th, and we need proper documentation for when we unleash 14 developers on the code in November. If we need to push some more time to you for that to do that, that’s what we’ll do. Obviously point 2 is also important eventually, but it can wait. However, depending on the amount of time it would take, it might be wise to just allocate for that too since there is also a certain cost to starting and stopping in the overhead time it adds.

How much more time would you need to:

  • Finish 1 and 2
  • Also finish 3
#12

I’d say another 10 hours to finish 1+3 (editing and docs), and another 10 hours on top of that to finish 2 (proper auth).

In that estimate for point 2 (auth), I’m not including a system for allowing users to be prompted to accept when someone wants to make them guide or realizer of a need or responsibility. That would probably take more time. But I could make it so that you can transfer a need or responsibility you own without the receivers approval, or just allow reality admins to do it.

#13

Ok, let’s do both. Start with 1+3 and then do 2.
We’re allocating another 20 hours.

#14

Awesome! I’m on it :heart:

Grand visions: How do we tie things together?
#15

Updating guide and realizer for a need or responsibility is deployed. Now on to adding and removing dependencies.

#16

Updating dependencies is deployed!

I still haven’t started point 2 and 3 from the other day’s list and I’m already at 92 hours (12 of the extra 20 already spent). I keep running into difficulties with either the neo4j-graphql-js library or neo4j itself when I write my own cypher queries.

For example, neo4j-graphql-js automatically generates mutations to add/remove relationships defined with the @relation directive in the schema, but I had to swap out the @relation directives for the more raw @cypher directive since the @relation directive doesn’t allow us to filter soft-deleted nodes. That meant I had to write my own queries for updating relationships.

Since I’ve had to skip the neo4j-graphql-js library and write my own queries in many places, I’ve also started running into problems when those queries don’t return every related node that the frontend might have asked for in the posted graphql query. For example, I might have a mutation that adds a Responsibility as a dependency to a node. The qraphql mutation returns the node and the responsibility. In graphql, I might want to also return the nodeId for the Need that the Relationship fulfills. That’s easy enough to define in the graphql mutation, but it means that when I write the cypher query for that mutation in the backend I have to make sure to fetch the FULFILLS relationship for the Responsibility. Now what if I also want to fetch the guide and realizer for the Responsibility, etc, etc. It quickly gets complicated.

I think we’ll need to refactor the backend and write our own resolvers (and do away with the neo4j-graphql-js library) as we continue to expand the graphql schema. That way we can set everything up so that a graphql query like the one above automatically fires off several, simple cypher queries that puzzle together the requested data. That’s easy enough to do with the graphql js library on the backend if we do away with neo4j-graphql-js. Maybe it’s also possible to do something similar while reusing stuff from neo4j-graphql-js, but we’d have to research that a bit. The library is still very new, so there might be undocumented features.

Sorry for the dense paragraphs, but I wanted to capture this here while it was still fresh in my mind.

Next steps:

I’ll start documenting everything in the README files to make it easier for people to contribute to the project. I’m also considering if I should spend an hour or two to add the ability for users to set the “name” property for their own user in the database. Right now people who sign up are only identified by their email, although everything is built to work if a Person’s “name” is set. What do you think @hugi? Would that be a good feature to have for tomorrow’s workshop?

Realities Release 0.1 target
#17

PS. I went ahead and added the ability for logged in users to edit their own name. That’ll make it easier for people to search for users tomorrow when assigning them as guides and reality admins. It took one hour. Hope that’s ok! :heart:

#18

@hugi Do you have everything you need for tomorrow’s workshop?

#19

Yes! This is fantastic work Erik! Thank you for your skill and diligence. This will be very interesting to try out tomorrow.

Yes, low effort high reward. Low hanging fruit and adds a lot to the user experience. Thank you for just making the decision to do this.

I think you’re right in that neo4j-graphql-js is probably not mature or specialised for our purposes in the long run. Let’s spend some time thinking about this in the coming months.

Yes, that’s a great next step.

All in all, WELL DONE! :smiley:

#20

Thank you! It’s a really fun project to work on! :smile: