dcache
dcache copied to clipboard
dCache
Roles Necessary to Perform QoS Transitions and Implications for Macaroons
Hi,
we at DESY use QoS for a variety of user communities from customers using VOMS to login and IAM-Tokens to login to using Username+Password.
This variety caused us to rely on macaroons in order to present a single tool-set to work with QoS transitions. Until 9.2 this worked very well, we use any of the method listed above to generate a macaroon with which our users manage QoS.
With 9.2 came the requirement for a specific role to be able to manipulate QoS. This breaks our workflow since Macroons as of 9.2 so far do not contain information on roles. Thus, using only a macaroon our users are no longer able to change QoS levels of files.
We can adapt our scripts to always check for the type of login rather than using a single login script. This would work well with VOMS certificates due their rather long lifetime as most VOs issue proxies with at least 1 day lifetime. The current IAM-Tokens have a lifetime of some minutes. Using Username+Password is even worse, each QoS Transition would require a fresh login. Even using bulk, changing QoS levels will often trigger 10+ requests and for each a new login is needed. While manageable, I see issues with our users with that.
Is it possible to either add roles to macaroons, so that they reflect the regular login, or make Roles on QoS configurable?
Thanks a lot, Christian
Christian,
I am not understanding. You can furnish your multi map configuration with role that allows qos transition. IOW give every users that role. All you need to change is your mulltimap config file.
Dmitry
As I understand it, the problem here is that the macaroon itself contains limited information about the user who generated the macaroon; specifically, the macaroon contains the macaroon-generator's uid, primary gid, the secondary gids (if any) and the username. No role information is recorded.
This means that, when using a macaroon, the reconstructed identity would not have any roles, even if the user who generated the macaroon had roles asserted when creating the macaroon.
The reconstructed identity takes place within the door, and is independent of gPlazma.
What Christian is asking for is that, either:
- the macaroon caveats are updated so that they record any roles the macaroon-generating user has asserted,
- the requirement for a role for QoS management be made configurable.
As two final thoughts: if the first option is chosen (macaroons updated to contain role assertions) then:
- The macaroon should contain only the asserted roles (none of the non-asserted roles).
- The asserted roles contained within the macaroon should probably further require the client to explicitly request that those roles are asserted. This is equivalent to how roles work currently, with normal login.
Hi Dmitry and Paul,
thanks a lot for quick replies. The multimap setup works fine, if there is an easy way to map the login credentials through gPlazma e.g. the VOMS-proxy DN or id from an IAM Token. Unfortunately, a lot of our smaller communities lack both capabilities or only of a VOMS endpoint for however long it will still be supported (VOMS server that is). As a solution, we used the regular ldap login to create a macaroon. So Paul is kind of spot on.
Manipulating/changing macaroon might be a box we better not open since the repercussions might be extensive, so maybe dropping the requirement in QoS might be the simpler solution?
Another alternative for us could be using Kerberos Tokens, which according to Tigran should also work with WebDAV, but Tigran could not remember how to get it to set up. But then we would need to assign the role as well and we need the UID:GID as well as the role. But maybe it is possible to do so?
Thanks a lot, Christian
Christian,
You said "The multimap setup works fine". So does it work for you or not?
My suggestion is to add qos role to all mappings in multi map config.
Dmitry
Can you share the gplazma configuration (e.g. on slack) so i can advise you better.
As I understand it, the problem here is that the macaroon itself contains limited information about the user who generated the macaroon; specifically, the macaroon contains the macaroon-generator's uid, primary gid, the secondary gids (if any) and the username. No role information is recorded.
in the end you map username to something in mutlimap - this is where you just add qos role. What am I missing?
in the end you map username to something in mutlimap - this is where you just add qos role. What am I missing?
What you're missing is that the door doesn't send the macaroon to gPlazma, it processes it locally. Therefore, there is no invocation of the multimap plugin, and no way to introduce new principals.
One of the ideas when developing the macaroon support was that the macaroon should completely identify the user: no login step is needed.
Oh Gawd. This means role should be part of the macaroon..... Hey, look at the patches!
Hi Dmitry and Paul,
Paul's spot on, but I could understand if major changes to the Macaroons might require more development effort, removing the role requirement would be fine for us as well. Especially, since I don't know how I would assign roles via the multimap plugin for a normal LDAP login?
Thanks a lot for the quick help!
Christian
The role is there because users are not supposed to change QoS willy nilly because tapes are precious resource. At Fermilab the model is - only one authorized user is allowed to change QoS for other users data per request. (tapes are budgeted by experiments). So role is not going anywhere. That means that macaroon has to have this authorization key added.
I'm not arguing against updating Macaroons so that they also carry information about roles: this should be done.
However, what you describe ("only one authorized user is allowed to change QoS for other users") doesn't mean the AuthZ decision needs to use a role.
From my perspective, a role is an optional aspect of their identity that the user can choose to assert: a "hat" that they can wear or not wear. This is rather similar to "sudo". I'm in the sudoers list on my laptop, which means I could delete all the files in /usr directory. I'm protected against making this kind of mistake because I would have to use sudo command to gain root privileges. I use sudo quite rarely and I'm extra careful whenever I do.
Using a role for QoS transitions makes sense if we want this kind protection against mistakes: the user normally can't trigger these QoS transitions; however, when they assert their QoS role, they can.
@christianvoss knows better, but I don't believe there is the need for this kind of protection at DESY: simply limiting QoS transitions to specific people would be sufficient.
So you are arguing against role to be used for this purposes. I have read your argument and not feeling it. Sudo role has associated sudo list config who can be root and who can't. So some users have that role, some don't. IOW I find it convenient to use role for this purpose. And...... circling back .... this was discussed at the meeting, posed and reviewed. I suggest to connect to meetings.
Hi Dmitry and Paul,
apologies the recent update kept me out of the loop a bit. I would agree with Paul, doing a QoS operation is already kind of a special operation anyway? Normally you open/close files, changing QoS is typically not part of the usage patterns of our users. It would be closer to the old srm-bringonline praxis before starting you jobs. It is a dedicated operation. For us users decide to archive a file to tape, disk -> disk+tape, or no longer keep the copy on disk, disk+tape -> tape. This is typically not part of any workflows.
Important to know would by, how to give users the role in the first place. Yes, it works through multimap using VOMS credentials and probably OIDC as well, but what about someone who just uses his LDAP login. I think this will be the most common praxis for us. Even with all that Keycloak hassle about which Pauls know a lot more than I do. That is still a Magic to me.
If you use LDAP plugin, you still can use multi map plugin. The multi map can be used with any authentication plugin. It works with OIDC plugin as well. A user, regular user, can't decide that all of a sudden they want their data to tape. At least at Fermilab this is not the case. When you use LDAP plugin all multi-map plugin will be doing is matching usernames to the qos roles.
Even srm-bring-online is normally done via rucio and it is not in general user control. srm-bring-online is relatively harmless operation. User can clog up your system for months with stages, but at least they don't use up more tape.
That's true, for us it is mostly a case of having data no longer on disk. Most of our users could no work with pins, they forgot to repin their data triggering a large amount of stores. We're considering, that users can trigger a flush to tape for some of their data using QoS instead of opening a large number of tickets.
But leaving that aside, even if we configure LDAP with Roles (if you or one from the team could provide an example) that leaves the issue for the Macaroons. The alternative would be to basically push hard for OIDC credentials at DESY and code the Token-renewal into our scripts directly, while hoping that the sync between Keycloak and LDAP works correctly and keeping track of the roles in dCache independent of LDAP. Still, mapping the LDAP Plugin would work with groups similar to VOMS, wouldn't it?