[Feature] easy create transaction not bind vm and ledger

[Linuxpizi]

🚀 Feature

1. i want a easy way to create transaction, not like create transaction must bind vm and ledger, and then i can use snarkos beacon service rest api to broadcast it.
2. maybe i misunderstanding how to to easy create (offline)transaction, so can add some example code.

Motivation

i want implement security, easy and convenient tools to create offline transaction and then use snarkos beacon service rest api to broadcast it

Implementation

Are you willing to open a pull request? (See CONTRIBUTING)

[howardwu]

You can read this code to see how to do this. I used snarkVM at commit 4b18ba7 (which is currently a draft PR until this feature is stabilized).

[dependencies]
reqwest = { version = "0.11", features = ["json"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
#snarkvm = { version = "0.9.6" }
snarkvm = { git = "https://github.com/AleoHQ/snarkvm", rev = "4b18ba7" }
tracing = "0.1"

[profile.release]
opt-level = 3
lto = "thin"
incremental = true

[profile.bench]
opt-level = 3
debug = false
rpath = false
lto = "thin"
incremental = true
debug-assertions = false

[profile.dev]
opt-level = 3
lto = "thin"
incremental = true

[profile.test]
opt-level = 3
lto = "thin"
incremental = true
debug = true
debug-assertions = true

use std::env;

use snarkvm::prelude::*;
use snarkvm::client::Client;

/// The main function.
pub async fn account_transfer<N: Network>(receiver: Address<N>, amount_in_gates: u64) -> Result<()> {
    // Initialize the client.
    let client = Client::<N>::new("https://vm.aleo.org/api")?;
    trace!("Initialized the client");

    // Derive the view key.
    let private_key = env::var("ALEO_PRIVATE_KEY").expect("ALEO_PRIVATE_KEY must be set");
    let private_key = PrivateKey::<N>::from_str(&private_key)?;
    let view_key = ViewKey::<N>::try_from(&private_key)?;

    // Load the block height that contains the record.
    let block_height = load_latest_record_height(); // <- Change to the block height containing your record

    // Load the records.
    let records = load_records::<N>(&client, &private_key, &view_key, block_height, false).await?;
    // Filter for the largest record.
    let (_commitment, mut sender_record) = records
        .into_iter()
        .max_by_key(|(_, record)| ***record.gates())
        .expect("No records found");

    // Transfer the amount in gates to the receiver.
    let (block_hash, transaction, next_record) = match create_transfer(
        &client,
        &private_key,
        &sender_record,
        receiver,
        amount_in_gates,
    )
    .await
    {
        Ok(result) => result,
        Err(error) => {
            bail!("Failed to create transfer: {error}")
        }
    };
}


/// Loads the record from the network.
async fn load_records<N: Network>(
    client: &Client<N>,
    private_key: &PrivateKey<N>,
    view_key: &ViewKey<N>,
    block_height: u32,
    check_to_tip: bool,
) -> Result<Vec<(Field<N>, Record<N, Plaintext<N>>)>, Error> {
    // Determine the block range.
    let block_range = if check_to_tip {
        // Get the latest block height.
        match client.latest_height().await {
            Ok(latest_block_height) => (block_height.saturating_sub(1))..latest_block_height,
            Err(error) => {
                error!("Failed to get the latest block height: {error}");
                return Err(error);
            }
        }
    } else {
        (block_height.saturating_sub(1))..(block_height.saturating_add(2))
    };

    // Initialize a list of records.
    let mut records = Vec::new();

    // Scan for records.
    for (commitment, record) in client.scan(*view_key, block_range)? {
        // TODO: Filter for credits.aleo records only.
        // Compute the serial number.
        let serial_number = Record::<N, Plaintext<N>>::serial_number(*private_key, commitment)?;
        // Check if the record is spent.
        match client.find_transition_id(serial_number).await {
            // On success, skip as the record is spent.
            Ok(_) => continue,
            // On error, add the record.
            Err(_error) => {
                // TODO: Dedup the error types. We're adding the record as valid because the endpoint failed,
                //  meaning it couldn't find the serial number (ie. unspent). However if there's a DNS error or request error,
                //  we have a false positive here then.
                // Decrypt the record.
                let record = record.decrypt(view_key)?;
                // Add the record to the list.
                records.push((commitment, record));
            }
        };
    }

    Ok(records)
}

/// Creates a transfer transaction, and broadcasts it to the network.
async fn create_transfer<N: Network>(
    client: &Client<N>,
    private_key: &PrivateKey<N>,
    sender_record: &Record<N, Plaintext<N>>,
    receiver: Address<N>,
    amount_in_gates: u64,
) -> Result<(N::BlockHash, Transaction<N>, Record<N, Plaintext<N>>)> {
    // Prepare the inputs.
    let inputs = [
        sender_record.to_string(),
        receiver.to_string(),
        format!("{amount_in_gates}u64"),
    ];
    // Execute the program.
    let (response, transaction) =
        client.execute(private_key, "credits.aleo", "transfer", inputs)?;

    // Check the response contains my address.
    ensure!(response.outputs().len() == 2, "Expected 2 outputs");
    match &response.outputs()[0] {
        Value::Record(record) => ensure!(
            **record.owner() == receiver,
            "Expected receiver to be the first output"
        ),
        _ => bail!("Expected a record in the first output"),
    };
    let next_record = match &response.outputs()[1] {
        Value::Record(record) => {
            ensure!(
                record.owner() == sender_record.owner(),
                "Expected sender to be the second output"
            );
            record.clone()
        }
        _ => bail!("Expected a record in the second output"),
    };

    // Broadcast the transaction.
    let res = reqwest::blocking::Client::new()
        .post(format!(
            "{}/testnet3/transaction/broadcast",
            client.node_url()
        ))
        .header("Content-Type", "application/json")
        .body(serde_json::to_string(&transaction)?)
        .send()?;

    if res.status().is_success() {
        let mut confirmed_block_hash = None;

        // Check that it is confirmed.
        loop {
            match client.find_block_hash(transaction.id()).await {
                Ok(block_hash) => {
                    confirmed_block_hash = Some(block_hash);
                    break;
                }
                Err(error) => {
                    trace!("Transaction not confirmed yet: {error}");
                    tokio::time::sleep(std::time::Duration::from_secs(15)).await;
                }
            }
        }

        if let Some(block_hash) = confirmed_block_hash {
            info!("Transaction confirmed in block {block_hash}");
            Ok((block_hash, transaction, next_record))
        } else {
            bail!("Transaction not confirmed: {transaction}");
        }
    } else {
        bail!("Failed to broadcast transaction: {}", transaction)
        // TODO: Retry?
    }
}

[Linuxpizi]

Thank you very much, I will continue to pay attention~

[Linuxpizi]

hi, i read the code repeatedly, but not resolve my problem, the main problem for me is i can easy get my unspent record use my view key, but very hard construct transaction if offline environment. first, i want to know it is possible to create an offline transaction, or create transaction must access beacon service.

[hairtail]

hi, i read the code repeatedly, but not resolve my problem, the main problem for me is i can easy get my unspent record use my view key, but very hard construct transaction if offline environment. first, i want to know it is possible to create an offline transaction, or create transaction must access beacon service.

curious about how to get unspent record easily without a fully synced node (vm and ledger). in my opinion, all your records can only be decrypted with your viewkey. the question is: if you do not have a node, who can do this for you? would you like to send your viewkey to someone? so what's the meaning of privacy here? why not bitcoin

[hairtail]

@howardwu Hi, howardwu, i heard that there is a plan for aleo browser-plugin based wallet like metamask in ethereum. curious about the architecture😄

[Linuxpizi]

hi, i read the code repeatedly, but not resolve my problem, the main problem for me is i can easy get my unspent record use my view key, but very hard construct transaction if offline environment. first, i want to know it is possible to create an offline transaction, or create transaction must access beacon service.

curious about how to get unspent record easily without a fully synced node (vm and ledger). in my opinion, all your records can only be decrypted with your viewkey. the question is: if you do not have a node, who can do this for you? would you like to send your viewkey to someone? so what's the meaning of privacy here? why not bitcoin

sure, above picture first i will get my unspent record with my view key use service a with aleo full node, then send the unspent record to service b(offline) to create a sign transaction, finally resend to service a to broadcast, of course service a will parse and verify the transaction ~

[hairtail]

so service a knows everything! meaning of privacy?😅

[Linuxpizi]

so service a knows everything! meaning of privacy?sweat_smile

security is relative, service a only know a signed transaction, service b has the private key, only service b(offline) can signature my unspent transaction~

[hairtail]

yeah, i know only service b can sign a transaction, my point is that service a has the knowledge of all your transactions (it can decrypted all your transaction once you give your viewkey). not about security, i mean privacy! with ethereum, only you can spend your token, but everyone know how many your address have. if you send your viewkey to service a, now service a knows.

[Linuxpizi]

yeah, i know only service b can sign a transaction, my point is that service a has the knowledge of all your transactions (it can decrypted all your transaction once you give your viewkey). not about security, i mean privacy! with ethereum, only you can spend your token, but everyone know how many your address have. if you send your viewkey to service a, now service a knows.

yeh, always a service needs to know view key to get my unspend records, so about privacy based on infrastructure services，even if private key security is relative, if there is a better way, I am also happy to share~

[kikakkz]

@howardwu Hi, howardwu, i heard that there is a plan for aleo browser-plugin based wallet like metamask in ethereum. curious about the architecture😄

curious too. we cannot let everyone to maintain it's own blockchain node if aleo want to be mass adopted.

[howardwu]

Correct, there is much more coming on this front for privacy and usability.

For context, I expect most users to be executing token transfers and simple programs on their machines. Large programs (such as machine learning inference) and interactive programs (like real-time gaming) are likely to require delegation, and our goal is to support these use cases in a privacy-preserving manner.

Transactions today are already delegable, and our proof system is designed to support private delegation in the future. In the present model, delegation to third parties reveals your record for this 1 transaction that was delegated (not your entire account). Private delegation is currently research that will manifest in code over the coming year.

I'll suggest taking a look at my points about account privacy in this PR: https://github.com/AleoHQ/snarkOS/pull/2071#issue-1452493746

Problems

In most cases, client-side software should be trial decrypting records by tracking the last block height that the software checked for, and merely scan the delta since the last block height for new records. The current REST endpoints do not protect this user-journey as the endpoints can easily be made accessible by third-parties.

For some software, an initial one-time sync (from genesis to tip) may be desirable to ensure the history of the account has been retrieved. In these instances, users should either 1) trial decrypt client-side, or 2) delegate the request to a third-party in a secure enclave if performance is required. The current REST endpoints are unable to enforce either of these design goals transparently for the user.

Lastly, for some software, they require merely a subset of records that belong to the account. For example, a program may only care about records produced and consumed by the program itself, or a program may only desire records pertaining to a subset of known programs that are already encoded in the program. Again, in this instance, the current REST endpoints are unable to offer filtering with these requirements.

Solutions

The recommended approach to detecting records is from client-side software to fetch blocks and perform trial decryption client-side. This can be done by using the GET blocks endpoint to fetch blocks, and subsequently check block.records().filter(|record| record.is_owner(view_key)) on the client-side in order to prevent node operators and third-parties from learning user account data.

In order to support the other capabilities that are described as missing in the problems section, Aleo is exploring the development of a secure enclave service to offer users the ability to perform performant scanning. In addition, future work to enable better filtering capabilities will be contemplated as part of the design of state in snarkVM.

[howardwu]

@howardwu Hi, howardwu, i heard that there is a plan for aleo browser-plugin based wallet like metamask in ethereum. curious about the architecture😄

I heard about https://twitter.com/theleowallet which may be worth taking a look at (I have not used the product before).

[alipostaci2001]

great conv. thanks