risc0
risc0 copied to clipboard
Published
20 hours ago •
risc0
risc0
[BUG] 'failed to fill whole buffer' error when using k256 crate
Bug Report
When using the k256 crate - a pure Rust secp256k1 implementation - in the zkVM guest, the guest program executes OK, but prover.prove returns an error on the host side: failed to fill whole buffer
$ cargo run .
proving execution...
thread 'main' panicked at host/src/main.rs:28:10:
failed to prove: failed to fill whole buffer
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Following the backtrace, the error seems to originate from the ark-std crate, in src/io/mod.rs. I am currently waiting for risc0 to reinstall. Once it does I will provide a full backtrace.
If I comment out the two critical lines in the guest crate's code, the proof runs OK, except obviously the elliptic curve point output is not written to the journal anymore.
diff --git a/methods/guest/src/main.rs b/methods/guest/src/main.rs
index 78e7b81..a2a905f 100644
--- a/methods/guest/src/main.rs
+++ b/methods/guest/src/main.rs
@@ -25,8 +25,8 @@ fn main() {
// Compute the public key of the given input.
// Uncommenting these lines out will cause the host to panic.
- let point = secp256k1_ec_mul(&input);
- output[32..].copy_from_slice(&point);
+ // let point = secp256k1_ec_mul(&input);
+ // output[32..].copy_from_slice(&point);
let output_bytes: &[u8] = output.as_ref();
env::commit(&output_bytes);
$ cargo run .
proving execution...
receipt journal output:
hash: 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793
point: 000000000000000000000000000000000000000000000000000000000000000000
verifying receipt...
OK
Steps to Reproduce
- I followed the instructions here to install the risc0 tools and toolchain.
- I set up a simple ZK key-statement proof which asserts than a given hash preimage is also the secret key to some public key on the secp256k1 curve. The full code is here, but i'll also include the guest and host code below.
Guest
#![no_main]
#![no_std]
use k256::elliptic_curve::group::GroupEncoding as _;
use risc0_zkvm::guest::env;
use risc0_zkvm::sha;
use sha::Sha256 as _;
risc0_zkvm::guest::entry!(main);
#[allow(dead_code)]
fn secp256k1_ec_mul(secret: &[u8; 32]) -> [u8; 33] {
let seckey = k256::SecretKey::from_bytes(secret.into()).expect("secret key is invalid");
let pubkey = seckey.public_key();
pubkey.as_affine().to_bytes().into()
}
fn main() {
let input: [u8; 32] = env::read();
let mut output = [0u8; 32 + 33];
// Compute SHA256 hash, and write it to the journal as a public output
let digest = sha::Impl::hash_bytes(&input);
output[..32].copy_from_slice(digest.as_bytes());
// Compute the public key of the given input.
// Uncommenting these lines out will cause the host to panic.
let point = secp256k1_ec_mul(&input);
output[32..].copy_from_slice(&point);
let output_bytes: &[u8] = output.as_ref();
env::commit(&output_bytes);
}
Cargo.toml:
[package]
name = "provable_sha256_dlog"
version = "0.1.0"
edition = "2021"
[workspace]
[dependencies]
# If you want to try (experimental) std support, add `features = [ "std" ]` to risc0-zkvm
risc0-zkvm = { version = "1.0.1", default-features = false, features = [] }
k256 = { version = "0.13.3", default-features = false, features = ["arithmetic", "serde"] }
Host
// These constants represent the RISC-V ELF and the image ID generated by risc0-build.
// The ELF is used for proving and the ID is used for verification.
use methods::{PROVABLE_SHA256_DLOG_ELF, PROVABLE_SHA256_DLOG_ID};
use risc0_zkvm::{default_prover, ExecutorEnv};
fn main() {
// Initialize tracing. In order to view logs, run `RUST_LOG=info cargo run`
tracing_subscriber::fmt()
.with_env_filter(tracing_subscriber::filter::EnvFilter::from_default_env())
.init();
// Arbitrary input.
let input = [1u8; 32];
let env = ExecutorEnv::builder()
.write(&input)
.unwrap()
.build()
.unwrap();
// Obtain the default prover.
let prover = default_prover();
// Produce a receipt by proving the specified ELF binary.
println!("proving execution...");
let receipt = prover
.prove(env, PROVABLE_SHA256_DLOG_ELF)
.expect("failed to prove"); // <-------- CRASH HERE
// Parse the journal from a receipt.
let output: Vec<u8> = receipt
.receipt
.journal
.decode()
.expect("failed to decode journal output");
assert_eq!(output.len(), 32 + 33);
let hash = &output[..32];
let point = &output[32..];
println!("receipt journal output:");
println!(" hash: {}", hex::encode(hash));
println!(" point: {}", hex::encode(point));
// The receipt was verified at the end of proving, but the below code is an
// example of how someone else could verify this receipt.
println!("verifying receipt...");
receipt.receipt.verify(PROVABLE_SHA256_DLOG_ID).unwrap();
println!("OK");
}
Cargo.toml
[package]
name = "host"
version = "0.1.0"
edition = "2021"
[dependencies]
methods = { path = "../methods" }
risc0-zkvm = { version = "1.0.1" }
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
serde = "1.0"
hex = { version = "0.4.3", default-features = false }
cargo run .-> Panics.
Expected behavior
If the k256 elliptic curve code should work with zkVM, the proof should be constructed successfully.
If k256 is doing something wrong/disallowed by the zkVM, I should see some clear error telling me what/why so that I can help address it upstream.
Your Environment
- risc0-zkvm version:
0.20.1 - Rust version:
rustc 1.76.0 (07dca489a 2024-02-04) - Platform/OS: Linux (Debian 12)
Additional context
TODO:
- Does the prover work in dev mode?
- Does the prover work if no elliptic curve math is done, but k256 is still used? What if we only do finite field math (e.g. multiply the input secret) instead of point-scalar multiplication?
![linear[bot] avatar](https://avatars.githubusercontent.com/in/20150?v=4 "Published by a pub.dev verified publisher"){kind=small}
The full backtrace:
$ RUST_BACKTRACE=full cargo run .
proving execution...
thread 'main' panicked at host/src/main.rs:28:10:
failed to prove: failed to fill whole buffer
Stack backtrace:
0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1959:27
2: risc0_zkvm::host::api::ConnectionWrapper::recv
3: risc0_zkvm::host::api::client::Client::prove_handler
at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/risc0-zkvm-0.20.1/src/host/api/client.rs:510:47
4: risc0_zkvm::host::api::client::Client::prove
at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/risc0-zkvm-0.20.1/src/host/api/client.rs:87:21
5: <risc0_zkvm::host::client::prove::external::ExternalProver as risc0_zkvm::host::client::prove::Prover>::prove_with_ctx
at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/risc0-zkvm-0.20.1/src/host/client/prove/external.rs:55:23
6: risc0_zkvm::host::client::prove::Prover::prove
at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/risc0-zkvm-0.20.1/src/host/client/prove/mod.rs:65:9
7: host::main
at ./host/src/main.rs:26:19
8: core::ops::function::FnOnce::call_once
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
9: std::sys_common::backtrace::__rust_begin_short_backtrace
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18
10: std::rt::lang_start::{{closure}}
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:166:18
11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
12: std::panicking::try::do_call
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
13: std::panicking::try
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
14: std::panic::catch_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
15: std::rt::lang_start_internal::{{closure}}
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
16: std::panicking::try::do_call
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
17: std::panicking::try
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
18: std::panic::catch_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
19: std::rt::lang_start_internal
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
20: main
21: __libc_start_main
22: _start
stack backtrace:
0: 0x607063810686 - std::backtrace_rs::backtrace::libunwind::trace::hbee8a7973eeb6c93
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
1: 0x607063810686 - std::backtrace_rs::backtrace::trace_unsynchronized::hc8ac75eea3aa6899
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x607063810686 - std::sys_common::backtrace::_print_fmt::hc7f3e3b5298b1083
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:68:5
3: 0x607063810686 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbb235daedd7c6190
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:44:22
4: 0x60706383d000 - core::fmt::rt::Argument::fmt::h76c38a80d925a410
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/rt.rs:142:9
5: 0x60706383d000 - core::fmt::write::h3ed6aeaa977c8e45
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/mod.rs:1120:17
6: 0x60706380d95f - std::io::Write::write_fmt::h78b18af5775fedb5
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/io/mod.rs:1810:15
7: 0x607063810464 - std::sys_common::backtrace::_print::h5d645a07e0fcfdbb
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:47:5
8: 0x607063810464 - std::sys_common::backtrace::print::h85035a511aafe7a8
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:34:9
9: 0x607063811b37 - std::panicking::default_hook::{{closure}}::hcce8cea212785a25
10: 0x607063811899 - std::panicking::default_hook::hf5fcb0f213fe709a
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:292:9
11: 0x607063811fc8 - std::panicking::rust_panic_with_hook::h095fccf1dc9379ee
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:779:13
12: 0x607063811ea2 - std::panicking::begin_panic_handler::{{closure}}::h032ba12139b353db
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:657:13
13: 0x607063810b86 - std::sys_common::backtrace::__rust_end_short_backtrace::h9259bc2ff8fd0f76
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:171:18
14: 0x607063811c00 - rust_begin_unwind
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
15: 0x6070633c6805 - core::panicking::panic_fmt::h784f20a50eaab275
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
16: 0x6070633c6e43 - core::result::unwrap_failed::h03d8a5018196e1cd
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
17: 0x6070633d6638 - core::result::Result<T,E>::expect::hee9e0eeebe8726d0
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1030:23
18: 0x6070633d6638 - host::main::h0cfe23277e3581b0
at /home/user/src/zkpreimage/host/src/main.rs:26:19
19: 0x6070633cf2b3 - core::ops::function::FnOnce::call_once::h1e862edc13c75a5f
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
20: 0x6070633cf2b3 - std::sys_common::backtrace::__rust_begin_short_backtrace::hf3ca1823e35a112e
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18
21: 0x6070633c9be9 - std::rt::lang_start::{{closure}}::h5edd8c0ef85debd4
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:166:18
22: 0x607063806ad1 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h37600b1e5eea4ecd
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
23: 0x607063806ad1 - std::panicking::try::do_call::hb4bda49fa13a0c2b
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
24: 0x607063806ad1 - std::panicking::try::h8bbf75149211aaaa
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
25: 0x607063806ad1 - std::panic::catch_unwind::h8c78ec68ebea34cb
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
26: 0x607063806ad1 - std::rt::lang_start_internal::{{closure}}::hffdf44a19fd9e220
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
27: 0x607063806ad1 - std::panicking::try::do_call::hcb3194972c74716d
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
28: 0x607063806ad1 - std::panicking::try::hcdc6892c5f0dba4c
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
29: 0x607063806ad1 - std::panic::catch_unwind::h4910beb4573f4776
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
30: 0x607063806ad1 - std::rt::lang_start_internal::h6939038e2873596b
at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
31: 0x6070633d686c - main
32: 0x77d02a678d0a - __libc_start_main
33: 0x6070633c714a - _start
34: 0x0 - <unknown>
Running in dev mode:
$ RISC0_DEV_MODE=true cargo run .
proving execution...
WARNING: proving in dev mode. This will not generate valid, secure proofs.
WARNING: Proving in dev mode does not generate a valid receipt. Receipts generated from this process are invalid and should never be used in production.
receipt journal output:
hash: 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793
point: 031b84c5567b126440995d3ed5aaba0565d71e1834604819ff9c17f5e9d5dd078f
verifying receipt...
OK
Running with only scalar finite field arithmetic instead of EC point multiplication:
diff --git a/methods/guest/src/main.rs b/methods/guest/src/main.rs
index 78e7b81..d104bb4 100644
--- a/methods/guest/src/main.rs
+++ b/methods/guest/src/main.rs
@@ -1,7 +1,7 @@
#![no_main]
#![no_std]
-use k256::elliptic_curve::group::GroupEncoding as _;
+// use k256::elliptic_curve::group::GroupEncoding as _;
use risc0_zkvm::guest::env;
use risc0_zkvm::sha;
use sha::Sha256 as _;
@@ -11,8 +11,11 @@ risc0_zkvm::guest::entry!(main);
#[allow(dead_code)]
fn secp256k1_ec_mul(secret: &[u8; 32]) -> [u8; 33] {
let seckey = k256::SecretKey::from_bytes(secret.into()).expect("secret key is invalid");
- let pubkey = seckey.public_key();
- pubkey.as_affine().to_bytes().into()
+ let scalar = seckey.to_nonzero_scalar() * seckey.to_nonzero_scalar();
+ let bytes = k256::FieldBytes::from(scalar);
+ let mut out = [0u8; 33];
+ out[1..].copy_from_slice(&bytes);
+ out
}
fn main() {
$ cargo run .
proving execution...
receipt journal output:
hash: 72cd6e8422c407fb6d098690f1130b7ded7ec2f7f5e1d30bd9d521f015363793
point: 0076710908241588c29833b5cfdfd42440cfcce085d0b9a247124793a078423c47
verifying receipt...
OK
So in summary:
- Dev mode works fine.
- Scalar multiplication in
k256works fine. - It's just EC point multiplication in prod mode which seems to have issues.
I also tried using your fork of k256:
diff --git a/methods/guest/Cargo.toml b/methods/guest/Cargo.toml
index b74f270..71cd0d9 100644
--- a/methods/guest/Cargo.toml
+++ b/methods/guest/Cargo.toml
@@ -8,4 +8,18 @@ edition = "2021"
[dependencies]
# If you want to try (experimental) std support, add `features = [ "std" ]` to risc0-zkvm
risc0-zkvm = { version = "0.20.1", default-features = false, features = [] }
-k256 = { version = "0.13.3", default-features = false, features = ["arithmetic", "serde"] }
+# k256 = { version = "0.13.3", default-features = false, features = ["arithmetic", "serde"] }
+
+[dependencies.k256]
+git = "https://github.com/risc0/RustCrypto-elliptic-curves"
+tag = "k256/v0.13.3-risczero.0"
+default-features = false
+features = ["arithmetic", "serde"]
+
+[patch.crates-io.crypto-bigint]
+git = "https://github.com/risc0/RustCrypto-crypto-bigint"
+tag = "v0.5.5-risczero.0"
+
+[patch.crates-io.sha2]
+git = "https://github.com/risc0/RustCrypto-hashes"
+tag = "sha2-v0.10.8-risczero.0"
$ cargo run .
proving execution...
thread 'main' panicked at host/src/main.rs:28:10:
failed to prove: failed to fill whole buffer
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I'm not immediately sure. Using k256 without the patch, this should just be like any other standard Rust crate. It's definitely something I've definitely something I've done before (e.g. in our ecdsa example). The error itself is in the prover client that talks to the server subprocess. I've seen this before when the prover subprocess fails. I imagine it can also happen when there is a breaking protocol change and the client and the server are running different versions.
I just retried with [email protected] and the crash still occurs. OP revised with code and config files for v1.0.1
The crash seems to occur with any secp256k1 point multiplication operation regardless of which implementation i use. I have also tried using the secp256k1 crate (binds to libsecp256k1) in the guest zkVM, and I still get the exact same crash as if I had used k256 to perform the point multiplication.
Is there some kind of syscall or operation which these secp256k1 implementations are performing which is somehow causing a crash?
I believe I found a much simpler reproduction method.
Guest program:
#![no_main]
#![no_std]
use num_bigint::{BigInt, Sign};
use num_integer::Integer;
use risc0_zkvm::guest::env;
use risc0_zkvm::sha;
use sha::Sha256 as _;
risc0_zkvm::guest::entry!(main);
fn modpow(base: &BigInt, exp: &BigInt, modulus: &BigInt) -> BigInt {
let mut product = BigInt::from(1);
let mut r = base.clone();
for i in 0..10 {
if exp.bit(i) {
product = (&product * &r).mod_floor(modulus); // correct
}
r = (&r * &r).mod_floor(modulus); // correct
}
product
}
fn main() {
let input: [u8; 32] = env::read();
let mut output = [0u8; 32 + 33];
// Compute SHA256 hash, and write it to the journal as a public output
let digest = sha::Impl::hash_bytes(&input);
output[..32].copy_from_slice(digest.as_bytes());
let input_num = BigInt::from_bytes_be(Sign::Plus, input.as_ref());
let output_num = {
let modulus = BigInt::parse_bytes(
b"FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F",
16,
)
.unwrap();
modpow(&input_num, &(&modulus - BigInt::from(2)), &modulus)
};
output[33..].copy_from_slice(output_num.to_bytes_be().1.as_ref());
let output_bytes: &[u8] = output.as_ref();
env::commit(&output_bytes);
}
The above program works fine. But with this one small modification, it fails.
--- a/methods/guest/src/main.rs
+++ b/methods/guest/src/main.rs
@@ -105,7 +105,7 @@ risc0_zkvm::guest::entry!(main);
fn modpow(base: &BigInt, exp: &BigInt, modulus: &BigInt) -> BigInt {
let mut product = BigInt::from(1);
let mut r = base.clone();
- for i in 0..10 {
+ for i in 0..100 {
if exp.bit(i) {
product = (&product * &r).mod_floor(modulus); // correct
}