Development of a Circom Gadget standard library for Lurk

[tchataigner]

Description

When our users will want to implement some of their projects in Lurk they will need some basic building tools to be integrated within our library (at least until they are able to easily integrate some by themselves).

As the Circom community has already produced quite a lot of really useful circuits, I am all for creating a standard library of pre-integrated Circom Gadgets that can be used by our users.

Ideally, we should also properly document the process of integrating a new gadget to such a library so that any of our user will be able to do it.

Motivation

This idea comes from discussions with @huitseeker and @winston-h-zhang.

To implement our idea for Light Client we will need to be able to do operations around the different hash algorithms used by the chains of interest to us (e.g., SHA3-256 or Keccak-256).

As such circuit does not exist in any of the bell* repository, the quickest route to implement them would be to leverage our CircomCoprocessor and some existing circom circuits.

Current behavior

This section aims to bring everyone to the same level of knowledge around the current DevX around the CircomCoprocessor.

If I am a user, here are the current setup steps to get a Circom circuit to work with Lurk:

Pre-setup

1. Procure themselves a valid Circom circuit.

CLI

2. Import the Circom circuit within Lurk context

Implementation

3. Create a structure that will serve as a CircomGadget foundation.
4. Implement both into_circom_input and evaluate_simple for the given CircomGadget/circuit.
5. Create an enumeration with the Coproc trait to use at the instantiation of Lang.

For a concrete example please refer to the Sha-256 example.

Observed pitfalls

Two elements in the current implementation can create pain points to our users:

1. Import restricted to CLI: By restricting Circom circuit import to CLI we are saying that we expect our users to find and get their own circom circuit by themselves, which can clearly be a pain point in the developer experience.
2. Implement their own gadget logic: Closely related to the previous pain point. As the users are the one importing their circuits, they also need to implement the plumbing for the gadgets. This provides flexibility but comes at the cost of simplicity.

Along with a reported bug that needs to be tested & fixed if needed:

1. Arguments for Circom circuits: See #628 and #789

Solution exploration

The core concept of a solution would be to directly integrate static files related to the circuits directly in some crates / repository / cdn to make them available in Lurk.

Static files

When it comes to the static files related to a given circuit we only need to have the related *.wasm and *.r1cs. A pair of those files can be around 30 MB. Here are the solutions I thought of:

1. Include files directly in the crate: Easiest implementation but will bloat the library in the long run.
2. Load it at runtime from an external source a. Github repositories: We could create one or multiple Github repositories with our Circom sources, and have their compiled files available in their different releases. b. A CDN/dCDN: We could store the static files on a content distribution network. But as this involves increasing costs for us, this does not seem like a great idea.

Recommendation: We should aim for standardized repository structure and behavior from which we could download static files at runtime.

Rust glue code

For now, I believe that we should restrict the glue code implementation to our library? However, it should be possible to create a generic interface between some Rust crates and our library to allow for the Gadgets' repositories to not only contain Circom code, but also Rust crates containing the glue code along with any other bit of logic needed.

Requirements

The requirements that will follow will be for a proof-of-concept over a Keccak-256 implementation. If this PoC is convincing enough, then we could switch to a more generic implementation.

1. Define a basic Gadget repository structure
2. Implement it over a Keccak-256 implementation in Circom.
3. Implement the Gadget code over lurk-rs
4. Add arguments specification over CircomCoprocessor

Open questions

Circom binary

I have a few open question about the Circom binary that we use in Lurk.

Currently I can see that we have a hard-coded version in our CLI files.

• Is there a reason for it to be hard-coded or do we mean to use the latest?
• Should we allow for Circom gadgets to use different versions of Circom in the future?

Are you planning to do it yourself in a pull request?

Yes