Creating and fetching a Rekor entry has a dynamic top level key

[eddiezane]

Description

When creating a new entry in Rekor the response contains a dynamic top level key of the merkle leaf hash. Working with a dynamic top level key is difficult and this endpoint does not support creating multiple entries in a single request. We may want to move this into the object itself.

$ curl -X POST -H "Content-Type: application/json" --data-binary @hashedrekord.json https://rekor.sigstore.dev/api/v1/log/entries 

{
  "9b4c44b2f870e6b6745d0a48f9217b3fff9963b35aa849d91c18206d2bf7834f": {
    "body": "....",
    "integratedTime": 1650065823,
    "logID": "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d",
    "logIndex": 2021365,
    "verification": {
      "signedEntryTimestamp": "..."
    }
  }
}

This is also the case when retrieving an entry.

$ curl -sSL https://rekor.sigstore.dev/api/v1/log/entries/547e07c992e21cce38be9bc60bd66fd667e872843b3f1ed637a7407557a286f1 | jq '.'

{
  "547e07c992e21cce38be9bc60bd66fd667e872843b3f1ed637a7407557a286f1": {
    "attestation": {},
    "body": "...",
    "integratedTime": 1651529236,
    "logID": "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d",
    "logIndex": 2212893,
    "verification": {
      "inclusionProof": {
        "hashes": [...],
        "logIndex": 2212893,
        "rootHash": "3d17a17f2cf7d43d27ae3b001200f5a20108d5922436c861326b13ee80cc50d3",
        "treeSize": 2226001
      },
      "signedEntryTimestamp": "..."
    }
  }
}

cc @lukehinds

[lukehinds]

I would like to see this changed too, as the few times I have wrote clients it can be a challenge to marshal this.

I think what we can do here is create sub issues in each client implementation to verify they can accommodate this change and email a notification to sigstore-dev

@lkatalin do you know if this would affect the sharing API?

also cc @dlorenc / @bobcallaway

[bobcallaway]

Agreed that it's not ideal; the merkle leaf hash (which is used to compute the entry ID) isn't known until the server verifies the proposed entry and then canonicalizes it. This could be addressed as part of the gRPC re-work

[eddiezane]

Ah that makes sense. What about something like this then?

{
  "uuid": "9b4c44b2f870e6b6745d0a48f9217b3fff9963b35aa849d91c18206d2bf7834f",
  "entry": {
    "body": "....",
    "integratedTime": 1650065823,
    "logID": "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d",
    "logIndex": 2021365,
    "verification": {
      "signedEntryTimestamp": "..."
    }
  }
}

[lkatalin]

@lkatalin do you know if this would affect the sharing API?

I don't think it should impact sharding since we aren't changing the structure of the entryID or how we use that data internally - just the way it is represented to the client in this output, right? If I'm understanding this proposal correctly, it will really help because that top-level key has been troublesome when receiving data from Rekor.

[bobcallaway]

I talked a bit more with @haydentherapper today about this... while it is a completely reasonable request, fixing this would break most (if not all) clients out there today to fix this - so I think we should consider batching this up with other changes to minimize churn.