Issue with Edgeryders form

owen · June 21, 2020, 5:47pm

I’m trying out the original form James built to figure out some integration problem with the webkit.

I am getting access control errors while using the standalone form. Is this because there is a separate whitelist for http://communities.edgeryders.eu/multisite_account.json ?

I have set up a test installation here: https://webkitsandbox.netlify.app

This address is already on the whitelist - so it should in practice work, but I am getting these errors:

Fetch API cannot load http://communities.edgeryders.eu/multisite_account.json?accepted_gtc=true&accepted_privacy_policy=true&edgeryders_research_consent=true&requested_api_keys=edgeryders.eu&auth_key=<Hidden from topic>&email=owen2%40gmail.com&username=owen2_39&password=cC65kggIR5zR2bN due to access control checks.

It seems to read correctly from the .env file, which looks like this:

VUE_APP_DISCOURSE_USER_URL=http://communities.edgeryders.eu/multisite_account.json
VUE_APP_DISCOURSE_AUTH_KEY= Hidden from topic
VUE_APP_DISCOURSE_DOMAIN=edgeryders.eu
VUE_APP_DISCOURSE_TOPIC_URL=https://edgeryders.eu/posts.json
VUE_APP_DISCOURSE_CATEGORY=250

Any ideas? Thanks

owen · June 21, 2020, 6:10pm

A separate question for @matthias about the multisite_account_api_key.json endpoint.

If I call this once the user has logged in via SSO, what is the payload?

Currently I am following the manual:

https://communities.edgeryders.eu/multisite_account_api_key.json?hostname=edgeryders.eu

But the manual also suggests a session cookie is required, is this then passed in a url parameter to get the user API key?

A cookie is set on redirect after SSO login, it looks like this:

redirect=YWRtaW49ZmFsc2UmbW9kZXJhdG9yPWZhbHNlJmF2YXRhcl91cmw9aHR0cHMlM0ElMkYlMkZjb21tdW5pdGllcy5lZGdlcnlkZXJzLmV1JTJGdXBsb2FkcyUyRmNvbW11bml0aWVzJTJGb3JpZ2luYWwlMkYyWCUyRmMlMkZjYTk5ZmE3ZmZiMGIxZDNiNDE3NDgyNjFlZWQ4ZjY2NzY1NmUyNjk3LmpwZyZlbWFpbD1vd2VuZ290JTQwZ21haWwuY29tJmV4dGVybmFsX2lkPTQwNjcmZ3JvdXBzPXRydXN0X2xldmVsXzAlMkN0cnVzdF9sZXZlbF8xJTJDdHJ1c3RfbGV2ZWxfMiZub25jZT0xMjM0JnJldHVybl9zc29fdXJsPWh0dHAlM0ElMkYlMkZsb2NhbGhvc3QlM0E4MDgwJTJGJnVzZXJuYW1lPW93ZW4=

Need a bit more on this part to get it working.

matthias · June 24, 2020, 2:44pm

Well … a cookie is a cookie You send it as a cookie (), not as a POST or GET parameter. It is its own type of parameter, and gets sent with every request to the domain for which the browser has stored the cookie (here communities.edgeryders.eu). The SSO payload return value does not contain the cookie, instead it is directly received by the browser when logging a user in to communities.edgeryders.eu.

We have configured the server to allow cross-domain use of that authentication cookie. (Like the manual says: “This only works because we allow cross-domain use of the session cookie via the permit-api-cors plugin.”) So you only have to care about CORS whitelisting and doing a successful SSO login, and the system should care for the rest.

Here’s how the process works as a whole:

Get your site’s domain added to “CORS allowed origins” on communities.edgeryders.eu.
Initiate a SSO authentication in your JS application, via communities.edgeryders.eu.
During SSO login, the user has to enter their username and password on communities.edgeryders.eu. As a result, they get the authentication cookie served from that site. Their browser stores it for domain communities.edgeryders.eu, giving them an active session on communities.edgeryders.eu (in addition to the SSO session on other sites via SSO login).
From your JS application, send a request to https://communities.edgeryders.eu/multisite_account_api_key.json.
Since there exists a cookie for communities.edgeryders.eu, the browser sends it together with the request automatically.
That cookie authenticates your script’s request to multisite_account_api_key.json and the API call should be successful.
The API call returns the user’s Admin API key for the requested Discourse forum(s) and can use that to access them from your JS application under the user’s account.

If that does not work, let me know. We tested and confirmed that cross-domain use of the cookie is possible.

Finally, here’s a way to get an idea what this cookie actually looks like (though no need to know, as it should be managed automatically): you log in as your normal user @owen to communities.edgeryders.eu and then analyze what the browser sends for authentication, and how:

In the Chrome web developer tools, go to “Storage → Cookies” and see what cookies are there in logged-in state.
Delete the cookies one after the other to find out which ones are essential for being logged in.
You will find that the following cookie is the only essential one (here with a fake value, but it looks like this):
```
_t:"d1afe7345cd1f6389a0d2ab7792569"
```

matthias · June 24, 2020, 2:58pm

Yes, every Discourse site has its own CORS whitelist. I just added https://webkitsandbox.netlify.app to “Allowed CORS origins” on communities.edgeryders.eu now, so that part should work now.

The installation of James’ forms software is well documented here. About your question it says (emphasis mine):

4. Enable and configure CORS in Discourse

The following steps have to be done in both Discourse instances (SSO provider and SSO client).

[…] Add the CORS origin. Add the domain that will host this Edgeryders Form Vue.js application to the cors_origins setting via the Discourse admin panel.

owen · June 24, 2020, 7:36pm

@matthias - Thanks for the explanation, I understand the process better.

On the last try, the GET call to https://communities.edgeryders.eu/multisite_account_api_key.json returned the login page source code rather than the user API key. The call was made after being returned to the page following the login.

However, this was from a local server (not whitelisted) - so it may explain this issue.

As for James’ form, it is still returning this access control error - I have just tried it again:

Fetch API cannot load http://communities.edgeryders.eu/multisite_account.json?accepted_gtc=true&accepted_privacy_policy=true&edgeryders_research_consent=true&requested_api_keys=edgeryders.eu&auth_key=7bcaf5fcc05cadf5ec114177c8d3c087&email=owengot%40gmail.com&username=owen_100&password=mn3YzT4Qr5HmJLn due to access control checks.

Is the address on the whitelist for sure? Thanks again for your help.

matthias · June 24, 2020, 7:41pm

Yes, it’s certainly on the whitelist.

The best way to assist will be that you provide me with a cURL call of the failing request so I can reproduce this here. For that: in Firefox web developer tools go to tab “Network”, right-click on a request and choose “Copy → Copy as cURL”. It’s equivalent in Chrome AFAIK. Since that command may include a session cookie, send it by DM.

owen · June 24, 2020, 7:43pm

ok - two minutes.

owen · June 24, 2020, 7:54pm

Sent

matthias · June 25, 2020, 12:32am

Turns out that API endpoint has accidentally become access protected somehow.

Because it works when including a session cookie, as follows:

curl 'https://communities.edgeryders.eu/multisite_account.json?{GET params as usual here}' -H 'Cookie: _t=bbeb……4a86'

(If that kind of command does not work like that for you, the API endpoint might now even be only accessible for admins. Didn’t test for that.)

With such a command, I don’t get the exact same error message when using curl, but I am served a document saying “You are being redirected”, with a link to the login page. Probably JavaScript recognizes from some response metadata that it’s an access protected page, and then tells you “Fetch API cannot load […] due to access control checks.” So this is not anymore about CORS.

We didn’t change the API endpoint, but we updated Discourse on the all sites except edgeryders.eu to the latest version a few days ago (and that broke other things already).

We’ll get the issue about this API endpoint fixed ASAP, within 1-2 days (now tracked as #241). Until then, you could add the cookie header equivalently to the curl command above to be able to work with this endpoint. Of course before live use, the issue has to be fixed as you’d not want to have an auth cookie of your own communities.edgeryders.eu login inside the software.

(In other news, I just refined the API manual sections about multisite_account.json and about multisite_account_api_key.json.)

owen · June 25, 2020, 10:26am

Thanks @matthias

daniel · June 27, 2020, 3:18pm

@owen This is now fixed, the endpoint is now accessible again.

owen · July 26, 2020, 4:25pm

@matthias @daniel - I am still getting CORS issues, from https://webkitsandbox.netlify.app/:

Access to fetch at 'https://communities.edgeryders.eu/multisite_account.json?accepted_gtc=true&accepted_privacy_policy=true&edgeryders_research_consent=true&requested_api_keys=edgeryders.eu&auth_key=HIDDEN&email=owengot%40gmail.com&username=sdasd_77&password= HIDDEN' from origin 'https://webkitsandbox.netlify.app' has been blocked by CORS policy: The 'Access-Control-Allow-Origin' header has a value 'https://bio26.edgeryders.eu' that is not equal to the supplied origin. Have the server send the header with a valid value, or, if an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I have double checked the configuration and the instructions and I believe there is nothing wrong on my end… Can you check why this is happening?

daniel · July 27, 2020, 1:57pm

I’m not able to reproduce the error. In the backend cors origin was allowed from “https://webkitsandbox.netlify.app/”. I removed the trailing “/” which might not be valid - at least for requests ( https://stackoverflow.com/questions/38103440/testing-cors-with-the-postman-tool-curl-in-chrome-console ). If you still get the error please post a curl request that triggers the error. I already tried requests with Postman but they seem to work fine.

owen · July 27, 2020, 5:11pm

@Daniel - the account creation part now works, however I am getting a 403 error calling posts.json (forbidden) that prevents the topic being created - pasted as a curl request below.


curl 'https://edgeryders.eu/posts.json' \
  -H 'Connection: keep-alive' \
  -H 'Api-Key: 111977e6...' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_16_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: */*' \
  -H 'Origin: https://webkitsandbox.netlify.app' \
  -H 'Sec-Fetch-Site: cross-site' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: https://edgeryders.eu/' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  --data-binary $'{"title":"Rethinking retirement - response by owen4","raw":"**We don\'t seem to discuss retirement much with others. How much do you think about your own retirement? What do you think about? And what do you avoid thinking about?**\\n\\nTest 1\\n\\n**How about work, hobbies, family life, leisure, pleasure... Which activities from your present life would you like to keep doing? Which things would you like to never do again?**\\n\\nTest 1\\n\\n**Great, now that you\'ve chosen what stays and what to leave behind - start to imagine your future self. The \'dream you\'. How do you get there? Which new skills and what knowledge do you want to acquire?**\\n\\nTest 1\\n\\n**Imagine your housing situation and the people around you (or their absence). Who are you surrounded by in your retirement?<br />Where and how would you like to live?**\\n\\nTest 1\\n\\n**If you know people who have retired, was there something to learn from their experience and approach? Could you share their impressions?**\\n\\nTest 1\\n\\n**How do you imagine sustaining yourself financially? Maybe you have a plan, a tip, or an idea, that you would like to share?**\\n\\nTest 1\\n\\n\\n\\n**name:** owen4\\n**age:** 31","category":"250"}' \
  --compressed

daniel · July 27, 2020, 5:04pm

Good. The api key is active. To create a post send it as a post request like curl -X POST 'https://edgeryders.eu/posts.json ... See https://docs.discourse.org/#tag/Posts/paths/~1posts.json/post for required parameters. To create a post e.g. topic_id is required.

matthias · July 27, 2020, 5:13pm

(Edited your post to hide the API key since I did not know if that’s @owen’s own key or just a throw-away key. In any case ideally do not post API keys publicly but rather in DMs, and just post the first few characters publicly.)

owen · July 27, 2020, 5:58pm

@daniel Thanks - unfortunately I’m getting the same 403 and I’m not sure why…

The payload looks like this:

{"title":"Rethinking retirement - response by owen","raw":"test_post 1","category":250,"topic_id":19987}

topic_id is there, so I don’t believe this is the issue.

curl 'https://edgeryders.eu/posts.json' \
  -H 'Connection: keep-alive' \
  -H 'Api-Key: 0e82...' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_16_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: */*' \
  -H 'Origin: https://webkitsandbox.netlify.app' \
  -H 'Sec-Fetch-Site: cross-site' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: https://edgeryders.eu/' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  --data-binary '{"title":"Rethinking retirement - response by owen","raw":"test_post 1","category":250,"topic_id":19987}' \
  --compressed

daniel · July 27, 2020, 6:30pm

test_post 1

daniel · July 27, 2020, 6:32pm

It needs to be a post request like curl -X POST - then it shall work.

owen · July 27, 2020, 6:42pm

fetch("https://edgeryders.eu/posts.json", {
    method: 'post',
    headers: { 'Api-Key': apiKey, 'Content-Type': 'application/json' },
    body: getPayload()
  })

this is the javascript equivalent of curl -X POST… unless I’m missing something?

the same function was working back in November… but I can try it with a different http request.