invidious
invidious copied to clipboard
[RFC] Authentication and Authorization API
Related: #427, #469
For various applications it is desirable to be able to read or write user data. This proposal describes a format for tokens, a method for authenticating requests, and endpoints for authorizing tokens.
Token format
Each token is a JSON object with the following schema:
{
"session": String,
"expire": Int64?,
"scopes": Array(String)
"signature": String
}
Tokens will be created with a given session
, which allows them to be revoked by the user.
Tokens may be issued with an expire
timestamp after which the token will be considered invalid.
Tokens will have a list of one or more scopes
determining their permissions. The format for scopes
is described below.
Tokens will have a signature
to ensure integrity of the other fields.
Example token:
{
"session": "v1:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"expires": 1554680038,
"scopes": [
":notifications",
":subscriptions/*",
"GET:tokens*",
],
"signature": "f//2hS20th8pALF305PJFK+D2aVtvefNnQheILHD2vU="
}
Authentication
All sub-routes of /api/v1/auth
must be authenticated.
Endpoints requiring authentication are authenticated with an Authentication: Bearer <token>
header.
Tokens are validated by taking all key-value pairs, minus signature
, and creating the following string:
key1=value1
key2=value1,value2,value3
Values are alpha-sorted. Keys are alpha-sorted and joined by newlines. There is no trailing newline.
The signature
is created by signing the string using SHA256-HMAC with the instance's HMAC_KEY
and Base64 encoded.
Using the above token as an example:
expires=1554680038
scopes=GET:tokens*,:notifications,:subscriptions/*
session=v1:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Signed using SECRET_KEY
provides the signature f//2hS20th8pALF305PJFK+D2aVtvefNnQheILHD2vU=
.
Scoping
In order to provide a means of limiting permissions for an application, tokens must provide one or more scopes
.
Each scope
refers to a sub-route of /api/v1/auth
. Scopes are defined as zero or more HTTP methods (semicolon-separated), colon (:), followed by an endpoint and optional wildcard for matching sub-routes.
A scope X is said to be containing
scope Y if X can match Y, but Y cannot match X. For example, :subscriptions*
contains GET:subscriptions/subscribe
.
Example scope allowing access to /api/v1/auth/subscriptions
using any method:
{ "scopes": [":subscriptions"] }
Example scope allowing access to /api/v1/auth/subscriptions
and any sub-routes using any method:
{ "scopes": [":subscriptions*"] }
Example scope allowing access to any sub-routes of /api/v1/auth/subscriptions/
using only GET and POST:
{ "scopes": ["GET;POST:subscriptions/*"] }
Example scope allowing access to any endpoint using any method:
{ "scopes": [":*"] }
A SID cookie is equivalent to a token with "scopes": [":*"]
.
POST /api/v1/auth/tokens/register
The /api/v1/auth/tokens/register
endpoint would support the following body (Content-Type: application/json
):
{
"scopes": Array(String), // List of scopes (comma-separated)
"callbackUrl" : String?, // URL for redirecting after successful authorization, optional
"expire": Int64? // Int64, optional
}
Calls to /api/v1/auth/tokens/register
must already be authenticated in order to receive a new token. This can be accomplished by a signed in user or with a Bearer
token with a scope containing POST:tokens/register
.
Calls made using a Bearer
token can only create tokens with equivalent or subsets of their own scopes.
If calls are made using a SID cookie, the user will be presented with a page presenting the requested scopes, and must then authorize the request before being redirected to callbackUrl
. If no callbackUrl
is provided, then after authorizing the token the user will be redirected to a page on the instance containing the newly created token.
If a callbackUrl
is provided, then after successful authorization the instance will redirect to callbackUrl
with ?access_token=<token>
set as the query.
POST /api/v1/auth/tokens/unregister
The /api/v1/auth/tokens/unregister
endpoint would support the following body:
{
"session": String?
}
Tokens can be unregistered by calling /api/v1/auth/tokens/unregister
. A token must have a scope containing POST:tokens/unregister
in order to unregister itself. In order to unregister other tokens a token must have a scope containing GET:tokens
and POST:tokens/unregister
.
If the response body is empty, or session
is not provided the token is assumed to be unregistering itself.
Users would also be provided a /list_tokens
endpoint for viewing and degistering tokens they had authorized.
Example endpoints
All endpoints below are sub-routes of /api/v1/auth
.
- GET/POST
/preferences
- Would allow GET of current preferences, or POST with updated preferences - GET
/notifications
- Same as/api/v1/notifications
with support for?since=
, see #469 - GET
/subscriptions
- Shows list of currently subscribed channels by the user- POST
/subscriptions/:ucid
- Subscribe to givenucid
- DELETE
/subscriptions/:ucid
- Unsubscribe to givenucid
- POST
- GET
/tokens
- Lists tokens authorized by user- POST
/tokens/register
- POST
/tokens/unregister
- POST
Example of use
For an application with a shared pool of users, such as FreeTube or CloudTube, an example token could be:
{
"session": "v1:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"scopes": [":notifications", "POST:subscriptions/*"],
"signature": "fNvXoT0MRAL9eE6lTE33CEg8HitYJDOL9a22rSN2Ihg="
}
This would allow clients to use ?since=TIMESTAMP
to receive any notifications missed when offline, and subscribe
to new channels. DELETE:subscriptions/*
is not included here to prevent unsubscribing to channels that still may need to be tracked by other clients.
Technically, an application only has to subscribe in order for a desired channel to be tracked by the instance (see #469). It would be possible for a client to subscribe and then unsubscribe so that the subscriptions
for the given account (and corresponding feed) doesn't become too large.
POST and DELETE should be used for /subscriptions/subscribe
and /subscriptions/unsubscribe
, and similarly for /tokens/register
and /tokens/unregister
, respectively.
Just updated the above, DELETE is not used for /tokens/unregister
since DELETE with a body is unsupported in some HTTP implementations.
Added with 2a6c81a89dd64c00dff0b37af1c6637771aad27e. There are a couple changes from what's proposed above, and clarification on anything that's missing:
-
/api/v1/auth/tokens/register
supportsapplication/x-www-form-urlencoded
so it's possible to request a token using something similar to the following:
<form action="https://invidio.us/api/v1/auth/tokens/register" method="post">
<input type="text" name="scopes[0]" value="GET:subscriptions*">
<input type="text" name="scopes[1]" value="POST:tokens/register">
<input type="text" name="scopes[2]" value="POST:tokens/unregister">
<input type="text" name="callbackUrl" value="https://www.example.com/callback">
<input type="submit" value="submit">
</form>
Which will require the user to authorize the token as expected.
-
callback_url
was renamed tocallbackUrl
for consistency with the rest of the API. -
/list_tokens
was renamed to/token_manager
to match with/subscription_manager
, and allows a user to revoke API tokens from the UI -
POST to
/api/v1/auth/tokens/register
with aBearer <token>
and no callback URL will return the requested token as the response body. Requesting with callback URL will return a 302 redirect. -
/api/v1/auth/tokens/unregister
will return 204 on successful response -
Content-Type: application/json
is required when POSTing with JSON body, otherwise you may encounter an unexpected result -
Bearer tokens will only be valid authentication for
/api/v1/auth/*
Currently the only API endpoints that require authentication are for tokens themselves. Unless there is any unexpected/unspecified behavior I'll close this and work on adding the other endpoints ( /preferences
, /subscriptions
, /notifications
, etc).
Added GET /subscriptions
, POST /subscriptions/:ucid
, and DELETE /subscriptions/:ucid
with 250860d92c166b06d4af5d713ec2640a5ad4e701.
Added GET /preferences
, POST /preferences
with 8a525bc13109e9bee3a59d3082d763f91b1dc7f6.
Please provide a web form for the registration of new tokens. I added this simple HTML to a file and opened that in the browser so I could easily create tokens for testing purposes, but it would be really nice if Invidious had its own page for this. Maybe link to this page from the token manager page?
<!-- TEMPLATE pre -->
<title>Invidious token generator</title>
<style>
label {
display: block;
}
</style>
<script>
let nextScopeIndex = 0;
function addScope() {
let ne = new ElemJS("label").text("Scope: ").child(
new ElemJS("input").attribute("name", `scopes[${nextScopeIndex++}]`)
)
let form = q("form");
form.insertBefore(ne.element, form.lastElementChild);
}
</script>
<!-- TEMPLATE header -->
<button onclick="addScope()">Add a scope</button>
<form action="https://invidio.us/api/v1/auth/tokens/register" enctype="application/x-www-form-urlencoded" target="_blank" method="post">
<input type="Submit">
</form>
<!-- TEMPLATE end -->
I cannot simply use curl to test because the initial token creation request redirects to a "confirm authorisation" web form with a CSRF token, and it would be very inconvenient to simulate the completion of that second form with curl.
It would be convenient if the token manager page also showed the list of scopes that the token is authorised to access, to allow me to easily tell tokens apart. Or, allow some sort of optional "label" to be provided during the token creation process, and display this label next to its token.
Added GET /subscriptions, POST /subscriptions/:ucid, and DELETE /subscriptions/:ucid with 250860d.
Since this is apparently the thread to be in:
cloud@future ~> curl -i -H 'Authorization: Bearer extremely_sexy_token' 'https://invidio.us/api/v1/auth/subscriptions/UC9-y-6csu5WGm29I7JiwpnA' -X POST
HTTP/1.1 204 No Content
Content-Type: application/json
Access-Control-Allow-Origin: *
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: default-src blob: data: 'self' https://invidio.us 'unsafe-inline' 'unsafe-eval'; media-src blob: 'self' https://invidio.us https://*.googlevideo.com:443
Referrer-Policy: same-origin
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Transfer-Encoding: chunked
(a long pause of over a minute without curl exiting, then finally prints this and exits)
curl: (18) transfer closed with outstanding read data remaining
Please provide a web form for the registration of new tokens.
You might try using GET /authorize_token?callback_url=https://example.com&scopes=SCOPE,SCOPE
although it's currently not well documented.
Or, allow some sort of optional "label" to be provided during the token creation process, and display this label next to its token.
Sounds good.
curl: (18) transfer closed with outstanding read data remaining
I'm having trouble reproducing this locally, testing both https://invidio.us
and https://dev.invidio.us
with curl --trace
:
https://invidio.us:
[snip]
<= Recv header, 28 bytes (0x1c)
0000: 54 72 61 6e 73 66 65 72 2d 45 6e 63 6f 64 69 6e Transfer-Encodin
0010: 67 3a 20 63 68 75 6e 6b 65 64 0d 0a g: chunked..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a ..
<= Recv SSL data, 5 bytes (0x5)
0000: 15 03 03 00 12 .....
== Info: TLSv1.2 (IN), TLS alert, close notify (256):
<= Recv SSL data, 2 bytes (0x2)
0000: 01 00 ..
<= Recv data, 0 bytes (0x0)
== Info: transfer closed with outstanding read data remaining
== Info: Closing connection 0
=> Send SSL data, 5 bytes (0x5)
0000: 15 03 03 00 12 .....
== Info: TLSv1.2 (OUT), TLS alert, close notify (256):
=> Send SSL data, 2 bytes (0x2)
0000: 01 00
https://dev.invidio.us
[snip]
<= Recv header, 28 bytes (0x1c)
0000: 54 72 61 6e 73 66 65 72 2d 45 6e 63 6f 64 69 6e Transfer-Encodin
0010: 67 3a 20 63 68 75 6e 6b 65 64 0d 0a g: chunked..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a ..
<= Recv data, 6 bytes (0x6)
0000: 31 0d 0a 0a 0d 0a 1.....
<= Recv SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1d .....
<= Recv data, 5 bytes (0x5)
0000: 30 0d 0a 0d 0a 0....
== Info: Connection #0 to host dev.invidio.us left intact
I'll try testing it out a bit more to see if there's an issue with Content-Length or some other bug with an empty response, although currently I suspect the issue is with HAProxy or some middlewhere somewhere. Probably better to open this as a new issue.
Appears to be a bug in curl, reported here: curl/curl#3968.
Added /api/v1/auth/feed
with 27e032d10dce0bd657e7b705d3dc8b24f1b55178.
POST
/api/v1/auth/tokens/register
[...] If calls are made using a SID cookie, the user will be presented with a page presenting the requested scopes, and must then authorize the request before being redirected to
callbackUrl
.
While it is certainly desirable to allow the user to control access to their account, this behaviour limits the capability of applications using the API to make authentication as painless as possible to the user. Especially in an environment where including or accessing a webbrowser is not possible, this makes it necessary to either store the credentials in the application (to allow for cookie-based authentication) or to have the user create the token on another device and then copy it over to the application in some way.
To circumvent this, it may be a good idea to take a page out of Google's book. To authorize the YouTube addon for Kodi to access features of a user's account, the addon requests a code from YouTube and then prompts the user to visit a webpage at google.com (formerly at youtube.com) and enter that code there. The webpage then allows the user to decide whether they want to authorize the addon to access the requested features.
It would be great if Invidious would include such a page as well as this would allow for, say, the development of a matching Kodi addon. :)
Note that this approach differs from that described by @cloudrac3r above in that the latter still requires the user to copy the token over to the application.
This will be amazing and will allow third party apps to be made for Invidious accounts such as NewPipe.
This issue has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely outdated. If you think this issue is still relevant and applicable, you just have to post a comment and it will be unmarked.
Bump
Any news on this?
@jxxp9 I'll see what I can do. I can't promise that this will be implemented soon, but I'm considering it!