Add support for searching through delegations more easily

[asraa]

The only way to search for a target in a delegation is by knowing for the target name ahead of time through client.GetTarget(name string).

If I want to perform a delegation search with the iterator to, let's say, find targets signed by a certain delegation name, or by finding targets that look like some/target/dir, or simply return all target names signed by the repository, I have to re-implement the code in [getTargetFileMeta]. However, c.loadDelegatedTargets is a private method, and is required to verifiably get the delegated targets file.

Question: TUF specification only describes "Fetching a desired target". How can I discover what targets the repository signs off on though?

Can we make the logic for a delegations iterator more general, and have a general client function for searching a repository targets based on some property? (e.g. match on target wild-card name, match on delegation name, match on delegatee name).

I admit I'm not sure on the best API for this. Maybe the answer is to make the iterator take in the client that can perform the validation logic internally, and users can use the iterator to do whatever they want.

[asraa]

cc @kommendorkapten @joshuagl for sigstore.

Right now, all we ever get from client.Targets() is the top-level metadata. We can search for client.GetTarget('rekor/rekor/rekor.0.pub)` but we don't know these apriori. Essentially right now all we're getting functionally is top-level targets.

[trishankatdatadog]

@asraa Would you please describe a few concrete queries here, so we can see how to implement this? Thanks!

[kommendorkapten]

What I imagine is something like this. The targets directory for this example:

$ find targets -type f
targets/rekor/key1.pub
targets/rekor/key2.pub
targets/rekor/foo
targets/another_delegate/bar

Adding a new function

type Filter func(t target) bool
FilterAll := func(t target) {return true}
func AllTargets(f Filter) []targets {
    // Perform a recursive search over all targets, including any targets defined by delegates, 
    // applying the filter for each target.
    return response
}

Then we could as an example define a filter for rekor:

func RekorOnly(t target) bool {
    return strings.HasPrefix(t.Path, "rekor/")
}

One could imagine a filter RekorKeys too, matching the path against rekor/*.pub.

Example queries could be: client.AllTargets(RekorOnly): [{target: rekor/key1.pub}, {target: rekor/key2.pub}, {target: rekor/foo}]

client.AllTargets(FilterAll): [{target: rekor/key1.pub}, {target: rekor/key2.pub}, {target: rekor/foo}, {target: another_delegate/bar}]

etc etc.

@asraa is this related to what you had in mind?

[joshuagl]

Question: TUF specification only describes "Fetching a desired target". How can I discover what targets the repository signs off on though?

TUF is designed under the assumption that it will be integrated into an existing system (where in more complex cases index/querying is already handled) and that we know what we are trying to retrieve securely with TUF ahead of time.

The semantics of delegations, especially hashed bin delegations, make query effectively impossible to provide in a TUF library API. How delegations are made (and what they mean) is application specific and forms part of the contract between the repository and the client.

Therefore, I don't think we can provide API for this in go-tuf. Hopefully we can provide API in go-tuf to make the application developer's life easier.

[joshuagl]

See also very relevant discussion in python-tuf https://github.com/theupdateframework/python-tuf/issues/1995

[kommendorkapten]

TUF is designed under the assumption that it will be integrated into an existing system (where in more complex cases index/querying is already handled) and that we know what we are trying to retrieve securely with TUF ahead of time.

This is a very good comment, as I think this was something I didn't really grasp during Sigstore TUF v2 client discussions. Or maybe I had an idea that the Sigstore client was the "existing system". Have to revisit this and think more when I get time.

[asraa]

Therefore, I don't think we can provide API for this in go-tuf. Hopefully we can provide API in go-tuf to make the application developer's life easier.

Yes! I think the issue here is that the access to delegated roles through the client isn't part of the publically exposed library: I have to recreate a DB, and the functions to create a DB, in order to search through delegations.

So looking at the library a little, I think these are some changes I'd like to make, and the proposed usage in the Sigstore client:

1. Discover delegations directly through the client. Maybe a method like client.Delegations() that internally runs the iterator and returns back a map/dictionary from (parent name -> child name) delegation relationships.

Right now a delegations iterator is called through getTargetFileMeta when searching for a particular given target. This internally uses the client's DB c.db so it is difficult to use the iterator publically.

2. Like the python-tuf issue: expose the delegation metadata, but perhaps given a delegation name. This would allow retrieval of the target names of a particular delegation.

It would be nice to have the delegation search baked in to the iterator, to avoid clients going off specification and using lower priority or continuing traversal on terminating delegations, so I also envision:

3. Search for wildcard matching targets: https://github.com/theupdateframework/python-tuf/issues/822

• Get All Targets (with priority) would be a * match. This would take into consideration that higher delegations have priority over lower ones, and replace them. It would return a list of TargetFileMeta (with the meta being associated to the "correct" delegator's metadata)

[jku]

Copying my latest comment from the python-tuf issue:

I guess I should update current thinking on this.

I think exposing the metadata to clients as described has security implications that may mean this is not a good idea. The fact that a delegated roles metadata contains targetpaths does not mean that those targetpaths have been delegated to the role. So exposing the list as is seems wrong, even if this is documented as unsafe.

The only really safe way to do this would be to, after choosing the role, to run the delegation lookup for each targetpath, and only expose the target to client if the targetpath really is delegated by the role in question. This sounds a bit wasteful but in practice might work just fine: in usual cases this would not lead to new metadata downloads and all required metadata would already be loaded in memory.

[rdimitrov]

Closing since the code base changed and this is no longer valid.

Thanks for raising this 👍