massa icon indicating copy to clipboard operation
massa copied to clipboard

Re-design datastore key queries (unify between JSON RPC API / GRPC / ABI AS / ABI WASMV1)

Open Leo-Besancon opened this issue 7 months ago • 0 comments

The JsonRPC API endpoint get_addresses lists, in the response, all the datastore keys of the given addresses. One issue with this is that there is no limits in the number of datastore entries for a given address. So if the address has a lot of keys, the response may be too large (e.g. larger than the default 50Mo response limit of the API) which causes issues.

For example, the following explorer page errors out because of it: https://explorer.massa.net/mainnet/address/AS1Ba1T2mMpHvLEhTvsNkSDqm4djno2ezVojGrtLf1wRd8ThfZD3 A request / response to the mainnet API showing the issue:

{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "get_addresses",
    "params": [["AS1Ba1T2mMpHvLEhTvsNkSDqm4djno2ezVojGrtLf1wRd8ThfZD3"]]
}
{
    "jsonrpc": "2.0",
    "error": {
        "code": -32008,
        "message": "Response is too big",
        "data": "Exceeded max limit of 52428800"
    },
    "id": 1
}

TODO:

  • [ ] List all the current ways we list datastore key (json rpc API, GRPC, assembly script ABI, WasmV1 ABIs, etc.)
  • [ ] Design a unified way to list the keys
    • Use (Bound, Bound) to define the range of keys to query. This is a way to generalize the optional prefix in some of the listing cases, with Bounds being Included, Excluded or Unbounded.
    • Limit the number of keys returned (max num would be a constant, or maybe a config parameter instead when applicable)
    • Best if non-breaking
  • [ ] Implement and test all the key listing cases

Leo-Besancon avatar Jul 08 '24 07:07 Leo-Besancon