I’m finally done with the frontend refactor/update and can now continue with implementing the last few features of the MVP. It’s all pushed to master and live at http://realities.theborderland.se/
In my last update in this thread I said that the only thing I had left to do was re-implement “creating needs and responsibilities”, but I had forgotten about the search feature. That took some time. A big part of the refactor was making sure that we use Apollo to fetch data from the backend in a sane way, rather than fetch everything from the database when the page loads and then refetch everything every time the user triggers a mutation. The previous search was built on the idea that we have the entire database cached in the frontend. Now it actually fetches results from the backend based on the search term entered by the user.
The things I have left to do are:
Deleting needs and responsibilities
Updating reality guide and realizer for needs and responsibilities
Adding and removing dependencies
I’m running out of time on my 80 hour estimate, but I think I can get all this done now that the code is in a better state.
The last couple of days I’ve worked on “soft deleting” (setting a “deleted” property on) needs and responsibilities. It turned out to be tricky since the neo4j-graphql-js library doesn’t have a system for filtering nodes that are fetched using their @relation graphql directive. I managed to get it working in the end but it drained a few hours. It’s pushed to master and deployed if you want to take a look at how I did it.
@hugi: Deleting nodes and responsibilities is merged to master, but without all the proper authentication. Right now anyone can do anything in the system as long as they’re logged in as a user. I only have 6h25m left on the budgeted 80 hours. What should I prioritise? Some options:
I can work on editing reality guide, realizer and dependencies as per the original plan, but I don’t think I’ll have time to implement all the proper authentication then. This is probably the best option for the unveiling on October 20.
I can work on getting all the proper authentication in place for the features we have now.
I can write better documentation for the stuff I’ve done so far, mainly focusing on guidelines for developing the frontend.
If you think it’s a good idea I could also keep working on the project for a while longer if there’s room to expand the budget by a few hours.
Points 1 and 3 are priority. We need point 1 to reach the MVP goal for the demo on the 20th, and we need proper documentation for when we unleash 14 developers on the code in November. If we need to push some more time to you for that to do that, that’s what we’ll do. Obviously point 2 is also important eventually, but it can wait. However, depending on the amount of time it would take, it might be wise to just allocate for that too since there is also a certain cost to starting and stopping in the overhead time it adds.
I’d say another 10 hours to finish 1+3 (editing and docs), and another 10 hours on top of that to finish 2 (proper auth).
In that estimate for point 2 (auth), I’m not including a system for allowing users to be prompted to accept when someone wants to make them guide or realizer of a need or responsibility. That would probably take more time. But I could make it so that you can transfer a need or responsibility you own without the receivers approval, or just allow reality admins to do it.
I still haven’t started point 2 and 3 from the other day’s list and I’m already at 92 hours (12 of the extra 20 already spent). I keep running into difficulties with either the neo4j-graphql-js library or neo4j itself when I write my own cypher queries.
For example, neo4j-graphql-js automatically generates mutations to add/remove relationships defined with the @relation directive in the schema, but I had to swap out the @relation directives for the more raw @cypher directive since the @relation directive doesn’t allow us to filter soft-deleted nodes. That meant I had to write my own queries for updating relationships.
Since I’ve had to skip the neo4j-graphql-js library and write my own queries in many places, I’ve also started running into problems when those queries don’t return every related node that the frontend might have asked for in the posted graphql query. For example, I might have a mutation that adds a Responsibility as a dependency to a node. The qraphql mutation returns the node and the responsibility. In graphql, I might want to also return the nodeId for the Need that the Relationship fulfills. That’s easy enough to define in the graphql mutation, but it means that when I write the cypher query for that mutation in the backend I have to make sure to fetch the FULFILLS relationship for the Responsibility. Now what if I also want to fetch the guide and realizer for the Responsibility, etc, etc. It quickly gets complicated.
I think we’ll need to refactor the backend and write our own resolvers (and do away with the neo4j-graphql-js library) as we continue to expand the graphql schema. That way we can set everything up so that a graphql query like the one above automatically fires off several, simple cypher queries that puzzle together the requested data. That’s easy enough to do with the graphql js library on the backend if we do away with neo4j-graphql-js. Maybe it’s also possible to do something similar while reusing stuff from neo4j-graphql-js, but we’d have to research that a bit. The library is still very new, so there might be undocumented features.
Sorry for the dense paragraphs, but I wanted to capture this here while it was still fresh in my mind.
Next steps:
I’ll start documenting everything in the README files to make it easier for people to contribute to the project. I’m also considering if I should spend an hour or two to add the ability for users to set the “name” property for their own user in the database. Right now people who sign up are only identified by their email, although everything is built to work if a Person’s “name” is set. What do you think @hugi? Would that be a good feature to have for tomorrow’s workshop?
PS. I went ahead and added the ability for logged in users to edit their own name. That’ll make it easier for people to search for users tomorrow when assigning them as guides and reality admins. It took one hour. Hope that’s ok!
Yes! This is fantastic work Erik! Thank you for your skill and diligence. This will be very interesting to try out tomorrow.
Yes, low effort high reward. Low hanging fruit and adds a lot to the user experience. Thank you for just making the decision to do this.
I think you’re right in that neo4j-graphql-js is probably not mature or specialised for our purposes in the long run. Let’s spend some time thinking about this in the coming months.
I took some time to work on other things this week after our recent deadline, but have started work on the documentation. I plan to finish off the last allocated hours on documentation and cleaning up our github issues during the coming week. That should prepare us for the upcoming hackathon and dedicated work week.
I just pushed an update to the main README file for the project and added a README for both the UI and API. I go over some higher-level concepts and patterns that we use in the code, the main libraries and services and some specifics that are not easily inferred from reading the code.
@hugi and others, could you take a look and give feedback by replying here? Is it good enough to help everyone get started at the hackathon / dev week?
If the docs are good enough I’ll spend a little bit of time cleaning up our issues. Right now we have a lot of issues that are half-finished since we’ve completed them without proper auth. I want to create new issues for proper auth so that we can move the ones that are half-finished now.
@hspeijer has started working on setting it up and is running into things that are probably good input to get the documentation up. We’re using the Gitter channel for faster back and forth chat. @erikfrisk, could you check in with him there?
Yes. First look is positive. I think I need to work on implementing something to get a feel for it and understand how it all fits together. I am not very good on getting this understanding by just reading the code.
