We are deploying a new onboarding funnel for edgeryders.eu. Multiple overlapping communities are working on the platform to understand and solve problems. A vital part of this work is to invite new stories and perspectives, and to do so we want to create a simpler and more welcoming first contact with our communities. Our way of engaging people revolves around asking open questions relating to the subject matter we are exploring.
Our new onboarding interface is a questionnaire where the answers get posted to edgeryders.eu, and the user would then automatically claim that content as their own when they registered on edgeryders.eu.
We are building the first version of it to be used on a tablet at the Biennale of Design in Ljubljana on November 14th. To leave some time for testing, the deadline to have the interface ready is on November 4th.
Project scope
Edgeryders have already built the API which creates a user on edgeryders.eu with a given email and makes a post on that user’s behalf. This API is described in the Edgeryders API documentation, under 3.4. Multisite account creation.
This brief is for creating a fully client-side javascript form which collects:
A users email
Responses to a number of questions from a user
Some additional but voluntary data
It then should then:
Compile answers to questions into a well-formatted Discourse post.
Each question should be in bold, followed by the answer.
Additional information, like age, goes at the bottom of the post.
Request a new account to be created with user email
Make a post with compiled answers on edgeryders.eu as a new user
App should follow the flow and logic described in the API documentation implementation example
The question definitions and the number of questions should be configurable via a configuration file in JSON or YAML format.
App should follow the error handling recommendations and display relevant error messages when they are returned from API and advice user on how to proceed
Once a post has been submitted, the app should show the account summary as described in API documentation implementation example
App should be completely client-side without any code running server-side, except for hosting.
App should be built with the Vue.js framework.
Developer must publish the code on GitHub and make regular commits and pushes during development.
Reporting and collaboration with Edgeryders must happen on edgeryders.eu platform.
App must be delivered by November 1st.
Licensing
This software will be open-source on the MIT license.
Bounty
We are offering 1000 EUR, paid on delivery.
How to apply
Post here, along with a link to some of your work.
Remember that it’s vital that you would be done by November 1st.
My name’s James, a former developer for Loomio, and have worked with Edgeryders in the past a bit on Loomio integrations and the Dreams platform. I’m currently employed by Optimal Workshop over here in New Zealand doing full stack development.
This piece of work is solidly within my skillset, and I have some space over the next few weeks to work on it, so the November deadline shouldn’t be a problem. I could start work on it next weekend.
This is great news. Since @gdpelican and I have worked together before with great results, I think we’ve found our developer. That was fast! Happy to be working with you again James. Feel free to start at your earliest convenience.
Hey @hugi, Angus’s suggestion here is a very good one; I’ve worked with both him and that plugin in the past, and both are great.
If you’d like, have a look and see if anything there might meet your use case; if so I can work with Angus to get a solution together based on his work so far (at reduced cost if it ends up being an easier build), or if not I’m happy to proceed with the proposed solution.
@Hugi, as we found our developer for this already, tell the Discourse Meta community that the call is closed?
No doubts about the quality of the plugin and its author … still I think it does not fit well into our software infrastructure. I will explain … but, first of all, hello and welcome from me I’m the guy in Edgeryders who has to try keeping all our IT infrastructure organized …
I’ll just list my reasons in no particular order and then let’s see if that persuades you:
Effort comparison. The Custom Wizard plugin would need the following extensions to work within our setup: (1) action type for user account creation at the end, as mentioned by Angus, (2) custom action implementation of that account creation action because we use SSO accounts so account creation has to be done remotely at the SSO provider side for which we now have a custom API endpoint. The custom wizard platform is a more generic solution, which is nice to have but also takes more effort to adapt (rule of thumb …). So the effort for both variants is comparable.
Core and periphery: what goes where. For this you need a bit of backstory: the previous (2012-2017) version of this web platform was built on, of all things, Drupal. When we made that decision, Discourse was not around yet, and we also initially liked the idea that admin users could adapt data types, forms, behavior and other aspects of the user interface using the admin interface, without having to do programming. That turned out to be a perfect recipe for a buggy mess of a software and we’re now really staying away from “user configurable forms and data types” inside our core platform. The core data of Discourse is posts and users, with the occasional custom user record field – nice and clean! Discourse and other “new wave single purpose” applications are great because they do one thing very well, instead of trying to do everything and failing spectacularly (like Drupal).
On a tangent: The failure of the “configuration over programming” approach taken by Drupal, at least as regards our use case, also means that I’m now much more inclined towards small, flexible software components that need programming for adaptations, rather than complex, generic components (like form builders) that supposedly need no programming but then always turn out to be just a bit too limited for a use case to really need no programming …
Now it certainly would not do harm to have one form builder plugin inside Discourse – it’s just that I try to establish this general architecture of keeping Discourse very close to its core function as a forum, while keeping peripheral data and features (forms, presentation websites, surveys etc.) to other independent web applications.
Stand-alone software or not? The Custom Wizard plugin has an unusual “semi-independent” software architecture: it is its own Ember application, but configurable from inside Discourse under the /wizard/ route. I understand where that comes from, as it’s a generalization of the Discourse setup wizard, which necessarily had to be a separate Ember application. (And we do a similar thing with our annotation system for Discourse by embedding a Rails engine into the Discourse interface under /annotator/.)
However for this use case, we anticipate quite several (>10) future deployments based on this form software over the years, most of which will be in close connection with project-specific small custom websites that have nothing to do with Discourse and are made using Vue.js. With that in mind, I think for us it makes sense to go all the way to independence: the survey / form software would only talk to Discourse by API, not sharing a admin user login or data storage with the Discourse database. It’s just the usual “high cohesion, loose coupling” principle of object oriented design, applied to applications …
Standardization. As an internal standard, we have Ruby (with Rails) for backend and scripting work and JavaScript (with Vue.js) for frontend work. I’m a huge fan of standards in IT, even if just within one organization: it helps with interoperability of components, finding and keeping developers, the economics of learning new technology, re-usability of the hosting setup on our server etc… That would be a reason to keep with the Vue.js approach for this little project, esp. given that there is previous work inside our organization to combine it with for a joint effort.
Now if something makes sense to just add to Discourse Core or an existing Discourse plugin, we’ll gladly deviate from our standard and use the Ember framework. Because then it’s a one-off project for an external developer, it will live in the upstream code base and we create no future legacy for our organization. That can’t work though if we expect that piece of software to be something we’ll work on and constantly expand though, such as in this case of the form-building component.
To sum up, I propose to not use the Custom Wizard Plugin for this project but rather go with the original proposal for a small independent Vue.js application, connected to Discourse by API.
Anyway, thank you for bringing that plugin to our attention. It definitely has its uses, and we may consider it for what we consider core functionality of this platform. (For example we use a form to collect consent to data usage in research before a user posts for the first time. With an added “show before posting” action, the Custom Wizard Plugin would be a good candidate to replace our current implementation of that feature.)
Excellent, thank you.
In this phase, it would be good to get the ux/ui designer on the platform @natalia_skoczylas.
On my end, it looks good and just as it should.
Alright, we’ve gotten to the point with this where the frontend is just about good to go and the big piece left is to put together a post for the Discourse API.
Was wondering if I could be granted access to an active auth_key (as mentioned in the API spec) for testing purposes (as well as some instructions on how to test it without spamming a real forum), as well as some thoughts on where to store the thing once this is deployed (since holding it in the clientside app won’t be secure)
I just sent it to you by direct message here on Discourse.
That’s a bit tricky. We could create a new “Communities” forum (a whole new Discourse forum that can be accessed by the same account), but our Discourse multisite installation is on a newer Discourse version than this very forum so it’s not the most realistic testing ground …
So I think let’s split up the testing into two parts:
Account creation. This API call does not create any notification spam, so no issue doing the testing on the edgeryders.eu forum. Use some pattern for the username that makes it easy for me to identify and remove the test accounts once the testing is done … such as surveytest_1, surveytest_2 and so on.
Posting to Discourse. After the account creation step, you’ll have the API key of a new Discourse user. Using that would indeed lead to notification spam, as a new user can only post to public categories at first. So instead, for testing use the Discourse API key of your own user (which I also sent in a direct message to you), and create the topic in the access protected category Survey Tests (category_id = 343) that I just created for this purpose. It’s only accessible to admins and to you, so there will be no notification spamming in the Unread / Latest / New topic lists or in e-mails.
A ApiKey (code definition) which is associated with a user and can be used to perform actions on behalf of this user (there also exists a ‘master api key’, which is associated with the Discourse system user and can do basically anything)
A UserApiKey (code definition) (usage spec) which the user can generate themselves for use with external applications (usually for mobile / desktop apps, or something without a server), and requires explicit permission from the user of scopes (read-only, write-only, one time password, etc.)
(NB that these two authentication systems are 100% distinct in Discourse the codebase)
Trouble is, as far as I can tell there’s not a way out of the box to use the first option from a clientside-only app, as the CORS settings on a Discourse instance will only accept values for User-Api-Key, which is linked to the second option. Here’s the code where the CORS settings are set, and a thread of someone having a similar issue, (in which the team’s response is 'Errr you need to be using UserApiKeys for this)
Using the second option seems possible, but would break the proposed workflow because the user needs to explicitly log in and grant permission to our application; I don’t think a hidden account creation followed by posting for them automatically is in the cards there. Also, it seems with this method the keys would come back encrypted with a public key that we provide, meaning that we’d need to store a private key (somewhere… probably not in the clientside app…) to decrypt it.
Options!
Backing the clientside app with a very simple node / ruby ( / whatever) server to perform API requests using “normal” API tokens ()
Modifying the Discourse source code to allow Api-Key creds to come in via CORS ()
Modifying the user workflow so that they explicitly
Complete the form
Are redirected to Discourse to create ( / log in to?) an account
Grant explicit permission to the form builder application to post for them
Are redirected back to the clientside app with a token we can use to post for them ()
Letting me know that I’m missing something simple and silly here ()
Ok this was unexpected … I guess I’m still too used to the 2017 Discourse with its one type of API access
Thanks for the detailed analysis!
So that could be just a proxy server with the only task to drop the Origin: header, so that Discourse sees the requests as not coming from a browser. It should work (and I think that’s what crossorigin.me does, but I did not check). But it’s a hack … so let’s see if we can find something better …
That would be the clean solution, right? I rarely disagree with the Discourse people on their software design decisions, but here I do disagree. The point of CORS is to protect the user from requests made by malicious sites that trigger actions on their behalf. Now the user is still protected if a Discourse site admin had to explicitly whitelist a site (that uses Admin API keys) via the “cors origins” setting because the Discourse site admin can check what that site does with the API key that is entered. And there are other legit reasons why a user might enter their own API key into a third-party website, beyond the edge case that we have here. For example a tool for API testing, or for data extraction etc…
Maybe we can persuade the Discourse devs and get this change into upstream. Otherwise we’d have to maintain it in our fork. In that case: not pretty, but clean.
Now our in-house Discourse developer is literally beyond the reach of the Internet for the next weeks, trekking around Annapurna. So, in the interest of progress: Would you want to try implementing this? For additional money of course. If so, please propose us a budget, or if that sounds a bit risky for this task, then first propose a budget for code exploration so you’d know the required effort better.
I’d install the code when ready. The deliverable would be a pull request against our Discourse fork, branch “stable”.
I’ve smacked together a little plugin that should do the trick here:
I’ll spin it up on a local instance this week to verify, and if so we should be able to get away with sticking that plugin onto the edgeryders.eu instance.
Like you say, it would be dangerous if you have e.g. Allow-Origin: * set, but since we’ll be whitelisting foreign domains it shouldn’t be too scary a thing (as long as you trust riot.im / any other domains in the whitelist etc to not abuse it)
If this works, it’s great! I would not have expected that change to be possible with a plugin. Let me know when you’re done testing and I should install it.
Alright, can confirm this will work for our use case; I do have to add a line (via this plugin) to PostsController to allow it to skip CSRF checking via the api:
One thing I wasn’t clear on was whether we are posting all of the responses in a single topic, or each response gets its own topic?
Next up: There’s CORS trouble with hitting the multisite endpoint as well (obviously 'localhost:8080 isn’t a great thing to put in your list of accepted origins), but I’m hoping I’ll be able to install that plugin locally as well and get an end-to-end test going soon.
and we’re now really staying away from “user configurable forms and data types” inside our core platform. The core data of Discourse is posts and users, with the occasional custom user record field – nice and clean! Discourse and other “new wave single purpose” applications are great because they do one thing very well, instead of trying to do everything and failing spectacularly (like Drupal).
)
)
)
)
… so let’s see if we can find something better …