[bug] Missing documentation of API authentication

[utrenkner]

Describe the bug with a clear and concise description of what the bug is.

The individual API calls are excellently documented. However, I am missing an important piece: How do I supply user or other needed information to the API.

Most API calls require authentication. I was able to create an auth token using the auth_flow.sh script. But still I am not able supply this token together with user(name) in a way the API understands.

Could you possibly add to the documentation one or several complete curl calls to the API so that I can pick up from that? Or even a step-by-step intro to the API authentication?

What's your GoToSocial Version?

0.5.2

GoToSocial Arch

amd64 (on FreeBSD 13.1-RELEASE)

Browser version

No response

What happened?

The API documentation says that a call like curl https://<myhost>/api/v1/accounts/verify_credentials should work without any parameters. But it answers:

Unauthorized: token not supplied

So, I add the token to the http header: curl -H 'Authorization: Bearer <mytoken>' https://<myhost>/api/v1/accounts/verify_credentials. But the API answers:

Unauthorized: user not supplied or not authorized

And this is where I am lost. I tried adding things like -H 'User: <myuser>' and similar. But without success.

What you expected to happen?

Above given error messages.

How to reproduce it?

No response

Anything else we need to know?

No response

[anantshri]

@utrenkner https://github.com/anantshri/gotosocial_bulk_follow/blob/main/README.md#note-about-auth-token might help. i had a bad time with that too. ultimately i used toolcli token in itself.

[drym3r]

It happens the same to me. @anantshr your script seems to do the same as published, it uses the bearer token (if I understood it correctly).

What do you mean when you said ultimately i used toolcli token in itself?

I've also tried posting with toot and the error was even weirder:

...
        <section class="error">
          <span>❌</span> <pre>Bad Request: json: cannot unmarshal string into Go struct field AdvancedStatusCreateForm.sensitive of type bool</pre>
        </section>
...

EDIT: I've been debugging a little bit more and I've discovered more stuff. All data is local, so don't panic if you see private data.

TL;DR: The problems seems to be related to the oauth key created manually via /api/v1/apps. It seems that it doesn't parse scopes nor add a default one.

Since Pinafore works correctly, I tried to post and see what it sends via the browser web tools. The post is correctly posted, as it can be seen.

The complete request is:

curl 'http://localhost:1081/api/v1/statuses' -X POST -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0' -H 'Accept: application/json' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Origin: http://localhost:4002' -H 'DNT: 1' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Site: same-site' -H 'Authorization: Bearer YWIXMTFHN2ETOTQ1ZC0ZMZLLLTKYZJATYJAWZWNKNMJHYTAZ' -H 'Connection: keep-alive' --data-raw '{"status":"Hello there!","visibility":"unlisted"}'

With this, I could reduce it to this:

curl 'http://localhost:1081/api/v1/statuses' -X POST -H 'Content-Type: application/json' -H 'Authorization: Bearer YWIXMTFHN2ETOTQ1ZC0ZMZLLLTKYZJATYJAWZWNKNMJHYTAZ' --data-raw '{"status":"Hello there!","visibility":"unlisted"}'

So, using Pinafora I could post and even copy the request as a curl and post it again, so the problem doesn't seem to be related to the endpoint.

The next is generate the token manually and test the last request with the new token:

active_url=http://localhost:1081

# get the cliend id and client secret
oauth_active=$(curl -sX POST -F 'client_name=Test Application' -F 'redirect_uris=urn:ietf:wg:oauth:2.0:oob' -F 'scopes=read write follow push' -F 'website=https://myapp.example' $active_url/api/v1/apps)
oauth_active_client_id=$(echo "$oauth_active" | jq -r ".client_id")                                                                                                                                       
oauth_active_client_secret=$(echo "$oauth_active" | jq -r ".client_secret")                  

# get the oauth token
oauth_token=$(curl -sX POST -F "client_id=$oauth_active_client_id" -F "client_secret=$oauth_active_client_secret" -F "grant_type=client_credentials" -F "redirect_uri=urn:ietf:wg:oauth:2.0:oob" $active_url/oauth/token)                                                                                                         

# print results so you can follow it
echo $oauth_active                                                                                                                                                                                        
echo $oauth_active_client_id                                                                                                                                                                              
echo $oauth_active_client_secret
echo $oauth_token   

Which returns:

{"id":"01TQHC1YSWYNQ84SRPDB9TAMYZ","name":"Test Application","website":"https://myapp.example","redirect_uri":"urn:ietf:wg:oauth:2.0:oob","client_id":"01EVXQS8GCR0W6X0BNFN55458F","client_secret":"c1005b38-5eef-45cd-9206-b7453d79f27a"}
01EVXQS8GCR0W6X0BNFN55458F
c1005b38-5eef-45cd-9206-b7453d79f27a
{"access_token":"NWY2MZZJOWMTMWIZMI0ZNDLHLWE4Y2YTMZU5MGNLNDZMNGQ4","created_at":1670358699,"expires_in":7200,"token_type":"Bearer"}

Then, I try the request that worked with the Pinafora token but using the token I just created:

curl 'http://localhost:1081/api/v1/statuses' -X POST -H 'Content-Type: application/json' -H 'Authorization: Bearer NWY2MZZJOWMTMWIZMI0ZNDLHLWE4Y2YTMZU5MGNLNDZMNGQ4' --data-raw '{"status":"Hello there!","visibility":"unlisted"}'

And returns:

...
        <section class="error">
          <span>❌</span> <pre>Unauthorized: user not supplied or not authorized</pre>
        </section>
...

The relevant logs of GtS are:

active   | timestamp="06/12/2022 20:34:32.690" func=security.(*Module).TokenCheck level=TRACE msg="authenticated client 01EVXQS8GCR0W6X0BNFN55458F with bearer token, scope is "

When posting from Pinafore, this is the generated log:

active   | timestamp="06/12/2022 21:18:05.493" func=security.(*Module).TokenCheck level=TRACE msg="authenticated user 01ZQJC0525NBEMKCWKKS85YYQF with bearer token, scope is read write follow push"

This is kind of weird, since the scopes are added to the request that creates the app. So the problem seems to be here.

I've tried to compare how the app are created in Pinafore, but it uses a different flow, using the endpoint /oauth/authorize.

To confirm the problem, when quering the database, it returns this:

017DSGC8FDMR89VQ9NRCBCD0JN|2022-12-06 21:08:54|2022-12-06 21:08:54|01XY6NDVFKYSM3T0R792E7V7X7|01ZQJC0525NBEMKCWKKS85YYQF|http://localhost:4002/settings/instances/add|read write follow push||||||MDNHOTCXOGUTODLHZC0ZODQ4LTLMYZYTNMQ3MDJJZTFLYWFK|2022-12-06 21:08:54.981357+00:00||||
01HAGKD6A8VJJ3K15A4BJNEV7B|2022-12-06 21:09:48|2022-12-06 21:09:48|01QWXNZ9PQ8NKHRBRW9HJ6SX2G||urn:ietf:wg:oauth:2.0:oob|||||||MJHINMMWNDMTY2NJYS0ZMMQWLTKYYZYTZJUYNWYWMZK3M2M4|2022-12-06 21:09:48.584465+00:00|2022-12-06 23:09:48.584478+00:00|||

They're quite different, but it seems that the main difference is the scope's column. As it seemed, it's empty in the manual token's row.

I'm not sure how to continue, I'm not used to the project's codebase, nor I'm an expert on Go. I've read https://github.com/superseriousbusiness/gotosocial/blob/ef5a9256a8fc35e644cc48734f39c84a8957de86/internal/api/model/application.go#L48 and https://github.com/superseriousbusiness/gotosocial/blob/1ede54ddf6dfd2d4ba039eb7e23b74bcac65b643/internal/processing/app.go#L35 and they seem correct, not much I can say about it.

Anyway, let's see if somebody can reproduce it and we'll see. I'm using the 0.6.0-rc3, btw.

[dennyabrain]

Hey all, I faced a similar problem and after struggling with it for a while, I was able to make authentication work. Should I add this documentation somewhere and make a PR for this? I think me and the OP are in similar boats and some beginner friendly documenation could help others.

[NyaaaWhatsUpDoc]

@dennyabrain that sounds like a good idea, would be much appreciated :blush:

[dennyabrain]

Great! will get to it. Do you have any recommendation on where I should add this? I see that the docs are generated from these markdown files - https://github.com/superseriousbusiness/gotosocial/tree/main/docs/api Should a create a new file called, lets say "authentication.md" or should I add it to the swagger.md file?

[9ssi7]

@dennyabrain Can you explain how you solved this?

[drym3r]

I've once again faced this issue when trying to update emojis. I didn't even remember this, i found it when searching on the repository. :sweat_smile: PD: @dennyabrain ping

[fyrfli]

I made significant progress on figuring this out today and working on creating a PR to update the documentation with that. stay tuned ...