Examples for authorization model

[RyanKadri]

📚 The doc issue

It looks like Release 0.11.1 added a token authorization system defaulted to on. I think it could be useful to give some suggestions for how users might use this system in real-world situations. I'm personally a bit unsure for my setup. For instance, if I'm deploying a containerized torchserve and interacting with it from another service, how would I use these tokens? By default, they're auto-generated and local right? I guess I could use a volume mount and try to get the information from these files to my torchserve consumer(s) but that sounds iffy especially since my containers may be ephemeral. Alternately, maybe the better approach would be to customize the token generation process to use some central secret store? It seems like that's possible but maybe it

Also, it seems like tokens expire. Not sure about the best way to coordinate that expiration across torchserve and a consumer without something pretty manual.

At a higher level, I see that this was released as a patch version change. I think this should be a major breaking change (from a version perspective) since setups that were working before will stop working and require manual intervention to fix. Was there a deprecation process that I missed (genuinely asking)?

Suggest a potential alternative/fix

No response

[agunapal]

Hi @RyanKadri Yes, we made the announcement pretty late https://github.com/pytorch/serve/issues/3184 but this was critical for us.

However, the easiest option for a user to default to earlier behavior is to disable it using --disable-token-auth

Here is an example of using token auth

https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md#configuration

[agunapal]

cc @udaij12

[karl-joan]

Hey, I am having the same thoughts. I would most definitely like to use token authorization but it remains a mystery to me how to securely distribute the tokens or how to perform authorization?

I'm running a TorchServe instance in a containerized environment. The current documentation seems to suggest accessing the key file and distributing it manually to other users/services. This approach doesn't seem scalable, especially with frequent container updates and multiple services making requests. Managing this manually adds significant overhead that's challenging to automate.

[agunapal]

Hi @karl-joan One possible solution is we provide an option to pass a json file with the keys, then in a k8s deployment, you can use the same file in all the pods. Would that work for you

[karl-joan]

So if I understand correctly, you mean it will possible to specify the keys in the key_file.json file with an external json? If yes, then this will work for me.

[agunapal]

Yes, that's right. cc @udaij12 @mreso

[mreso]

Yes, we should implement the possibility to

• [ ] Set token by user
• [ ] Have multiple tokens active in parallel
• [ ] Create and delete tokens via the api Cc @agunapal @udaij12

[karl-joan]

Hey @mreso, @agunapal! I was wondering if there is any update on this :smiley:

[agunapal]

Hi @karl-joan This will be addressed in 0.13.0 of TorchServe

[tadayosi]

A feature to pass a key_file.json not only helps in production, but also helps in testing where we want to automate containerised torchserve tests with the token authentication. Thanks.