metaboss

metaboss copied to clipboard

A simple ARWeave upload tool would be a nice to have addition to complete metaboss

I agree, and it's been on my mental to-do list for a while. It's quite a bit of work though, unfortunately, as there aren't great Arweave libraries for Rust. Caleb Everett recently wrote a Rust CLI for uploading to Arweave which you can use in the meantime and which I'll probably borrow ideas from at some point: https://github.com/CalebEverett/arloader.

It is a pile of work to do from scratch in Rust. Good news is that I've kind of already done it and hopefully set up the library in a way that you could use the main lib functions to add commands to your cli. If you have any questions about how it's organized, happy to help. On the other hand, the demarcation point of arloader is links to metadata files with a nice workflow for getting them updated with links to your assets. So you could run arloader and then get back a nice json file of links you would use to update your metaboss json objects. I'd also be happy to take that a step further and add a function to output json in metaboss format if you think that would be useful.

Just to shed a little more light on why it is such a heavy lift - the transaction format requires you to create merkle trees from chunks of your data and then hash them together with a specific algorithm. If your transaction sizes are less than 10MB, you can upload the full tx, but if bigger than that you have to upload in 256k chunks. Lastly, the new bundle transaction format, which is totally the way to go, has its own specific serialization format.

On Fri, Dec 3, 2021 at 10:44 AM Samuel Vanderwaal @.***> wrote:

I agree, and it's been on my mental to-do list for a while. It's quite a bit of work though, unfortunately, as there aren't great Arweave libraries for Rust. Caleb Everett recently wrote a Rust CLI for uploading to Arweave which you can use in the meantime and which I'll probably borrow ideas from at some point: https://github.com/CalebEverett/arloader.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/samuelvanderwaal/metaboss/issues/30#issuecomment-985746426, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEIMQ4F4TDLR4JGPDWWMBLUPEFYRANCNFSM5JHAV4MA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

These are a little sparse, but here are the docs on the lib https://docs.rs/arloader/0.1.30/arloader/

and of course I'd take issue with this ;)

as there aren't great Arweave libraries for Rust

These are a little sparse, but here are the docs on the lib https://docs.rs/arloader/0.1.30/arloader/

and of course I'd take issue with this ;)

as there aren't great Arweave libraries for Rust

I didn't realize you had broken this out into an actual library. This is great!

I did end up adding a command to export the items links that I understand the candy machine expects so may not be a bad workflow to use arloader and then take your items links json file and start using metaboss. But I also broke out all the commands into their own module if you wanted to include lower level functionality in your cli.

The only thing that may not be completely plug and play is the output format, which I thought was pretty slick and borrowed from solana_cli_output.

Inspired by this I built https://github.com/kevinrodriguez-io/harbor quite a while ago for uploading into arweave. It's super flexible.

@kevinrodriguez-io looks cool! A couple of things you might want to consider on the arweave upliad:

1. Unfortunately, you can't just rely on the presence of the file at the url as conformation that it has been successfully uploaded. Arweave aggressively caches everything, even failed transactions, so you have to check status and make sure confirmations are greater than 25.
2. You may want to consider implementing bundles to reduce the number of transactions by a couple orders of magnitude. Otherwise, you run the risk of failed tx, with for example a 10k collection, you would have 20k transactions and Arweave can only write 1000 every two minutes.
3. Or you could use bundlr and upload individual items to them and they do the bundling for you. You can also pay in native tokens, like SOL.
4. Cmv2 has a bunch of different upload options now.

That's great insight on the arweave uploader, I've been strongly considering moving to bundlr and requesting only a solana keypair. That would be the best. Right now arweave upload is a bit choppy but at it defends itself against empty txs (No blank nfts)