Bulk: pin request stuck in "running" state when using `root:` caveat and relative dir

[onnozweers]

Dear dCache devs,

A user has trouble staging a directory when using a macaroon with a root: caveat. This happens in these conditions:

• dCache 10.2.10, or the master branch at commit 7ce5214c16efe268fc5ed190ae7d402a7c38dadc
• Macaroon has a root: caveat
• Staging through the bulk-requests API call
• Staging a dir, not a file
• Specifying the directory as /mysubdir, relative to the dir in the macaroon's root:, omitting the parent path that is already in the macaroon.

Additional observations:

• Specifying the full path results in a normal operation.
• Using a macaroon with path: instead of root: also results in normal operation (perhaps because then you use a full path anyway).
• With root: and a relative path to a file instead of a dir, it also works as expected. The file appears in the restore queue.

Here is an example of the request that ends up stuck in "running" state:

curl --config /Users/onno/.ada/headers/authorization_header_6Xm0cWm6C3HS -H 'accept: application/json' --fail --silent --show-error --ipv4 -H 'content-type: application/json' -X POST https://dcacheview.grid.surfsara.nl:22880/api/v1/bulk-requests -d '{"activity": "PIN", "arguments": {"lifetime": "7", "lifetimeUnit":"DAYS"}, "target": ["/tape/test2"], "expand_directories": "TARGETS"}'

The directory contains two files:

[onno@db4 tape]$ ls -l /pnfs/grid.sara.nl/data/users/onno/tape/test2/
total 1
-rw-r--r--. 1 onno onno 14 Sep  1 14:11 1
-rw-r--r--. 1 onno onno 14 Sep  1 14:12 2

This is a stuck request:

% curl -H "Authorization: bearer $BEARER_TOKEN" -X GET "https://dcacheview.grid.surfsara.nl:22880/api/v1/bulk-requests/770b0316-3d3f-454c-b9a6-44affc21534e"
{
  "nextId" : -1,
  "uid" : "770b0316-3d3f-454c-b9a6-44affc21534e",
  "arrivedAt" : 1756737891336,
  "startedAt" : 1756737891601,
  "lastModified" : 1756737891601,
  "status" : "STARTED",
  "targetPrefix" : "/pnfs/grid.sara.nl/data/users/onno",
  "targets" : [ {
    "target" : "/tape/test2",
    "state" : "RUNNING",
    "submittedAt" : 1756737891339,
    "startedAt" : 1756737891339,
    "id" : 25045069
  } ]
}

When using a path: macaroon, or when using the full path instead of the relative, the request looks like below. As you can see, the directory itself is "skipped" but the files in it have been populated in the request.

% curl -H "Authorization: bearer $BEARER_TOKEN" -X GET "https://dcacheview.grid.surfsara.nl:22880/api/v1/bulk-requests/0a91dbe8-aa8c-4234-ab22-c3d580cc9e37"
{
  "nextId" : -1,
  "uid" : "0a91dbe8-aa8c-4234-ab22-c3d580cc9e37",
  "arrivedAt" : 1756737696610,
  "startedAt" : 1756737696808,
  "lastModified" : 1756737696808,
  "status" : "STARTED",
  "targetPrefix" : "/pnfs/grid.sara.nl/data/users/onno",
  "targets" : [ {
    "target" : "/pnfs/grid.sara.nl/data/users/onno/tape/test2",
    "state" : "SKIPPED",
    "submittedAt" : 1756737696832,
    "startedAt" : 1756737696613,
    "finishedAt" : 1756737696832,
    "id" : 25045063
  }, {
    "target" : "/pnfs/grid.sara.nl/data/users/onno/tape/test2/1",
    "state" : "RUNNING",
    "submittedAt" : 1756737696831,
    "startedAt" : 1756737696831,
    "id" : 25045065
  }, {
    "target" : "/pnfs/grid.sara.nl/data/users/onno/tape/test2/2",
    "state" : "RUNNING",
    "submittedAt" : 1756737696831,
    "startedAt" : 1756737696831,
    "id" : 25045066
  } ]
}

I would expect, that with a root: macaroon, a relative path would either work (preferably), or that it would fail with an error (if it really can't be supported). I would not expect that the request keeps hanging forever in the bulk service. So I assume there is some kind of bug.

Kind regards Onno

[onnozweers]

I just tested with the latest master snapshot: same problem.