[Feature] Fetch entire mapping

[benjaminpreiss]

🚀 Feature

It would be great if I were able to load an entire mapping into JS, to be able to query it locally.

Motivation

This would enable computations where one needs to know a large fraction of the mapping. E.g. weighted random number generation, where the weights are stored on-chain:

mapping weights: u64 => u8;

Then one can use walker's alias method locally (in js) to generate the necessary tables to perform efficient weighted random number generation.

If loading a mapping into JS is not possible, I'd have to do the computation on chain, which would mean at least a million computations for a mapping size of one million.

Implementation

I am not sure how an implementation would look like in snarkOS honestly.

[vicsn]

Thank you for sharing! Summarizing from the Telegram channel: this can indeed either be implemented at the snarkOS level - or explorers can add support for it. Let's leave this issue open to track progress on this front from different explorers or designs.

[raychu86]

We currently have an implementation of this in snarkOS via a REST endpoint:

GET /<network>/program/{programID}/mapping/{mappingName}?all={true}&metadata={true} https://github.com/ProvableHQ/snarkOS/blob/6d11788df01e1173f2c92e784d710d7c6bab9868/node/rest/src/routes.rs#L232-L233.

This is an expensive operation because a mapping can be enormous, so the route is gated behind the node's JWT. Currently, you will need to run your own node to access this JWT or you need to find a node operator that trusts you with their JWT and access to their node REST endpoint.

Explorer support for this is a separate conversation based on their capabilities to support these expensive calls. (Imagine trying to fetch the public balances of all addresses as a JSON via a REST api)