Manifest generation on demand

[glihm]

Is your feature request related to a problem? Please describe. The manifest is a file that is generated by sozo once the world has been migrated. The manifest contains all the resources (contracts/models/events) that have been deployed, the addresses, class hashes and ABIs.

However, currently the manifest if only generated one the world is migrated. Which implies that it can't be generated for a world already migrated.

An important thing to mention is that, in dojo, all addresses are deterministic, based on two values:

1. The class hash of the world (which is directly linked to the Dojo's version).
2. The seed chosen by the user in the dojo_<PROFILE>.toml.

Describe the solution you'd like The purpose of this issue is to discuss what could be the best approach to access the manifest on demand.

Here are the considerations:

1. If the manifest is generated, but something goes wrong during the migration, the manifest will not represent the actual onchain state of the world. But rather the "what would be" state, if migration is completed.
2. Generating the manifest on demand is possible using the two values described earlier, and before a sozo plan command was used. Currently sozo has the sozo inspect commands which actually pulls from the chain everything that is required to generate the manifest. We may add a sozo inspect --output-manifest /tmp/manifest.json option?

Additional context The manifest data are defined here: https://github.com/dojoengine/dojo/blob/baf0ce91be46a9c5c933feb2efe811067a8b7455/crates/dojo/world/src/diff/manifest.rs#L16-L21

[knownasred]

I think a variation of 2 would be a great option.

The manifest is now used for more than just storing the current state of the migration for comparison with the hoped state, it is also a way to provide information for the different SDKs (dojo.js)

It is also used for many scripts to extract the addresses of specific contracts: https://github.com/search?q=manifest_dev.json++%7C+path%3A*.sh&type=code&ref=advsearch

Generating the post-migration state is not useful for the use-cases defined above, and I am not convinced that using a command that can alter global state to modify local (on your computer) state is a good idea.

For the second solution, I would argue not to write the manifest to disk, but instead add a variant of --output-format json that would write the expected state to stdout in JSON format. By using stdout, you reduce steps needed for scripts, by only requiring a pipe to jq, instead of managing temporary files.

[glihm]

Thank you @knownasred for the comments here, let's move to that direction then.

About the output option, where does it make more sense to you?

Should it be using sozo inspect? Currently, sozo inspect is by default using the manifest. And if the manifest is not found, it fallback pulling data from the chain.

Technically speaking, when we run sozo build, we should be able to have all the information too. We could also think about moving the manifest output on build? But it makes it heavier as sometime we don't want to have the whole setup just to have a manifest.

IMHO, having sozo inspect --output-manifest which as you mentioned outputs the JSON to STDOUT could be the less intrusive and easiest way to get a manifest from a project without having anything to build.

sozo inspect is profile based too, so pulling different manifests will be as simple as adding --profile option to the sozo inspect command.

# Default profile
sozo inspect --output-manifest

# Sepolia
sozo inspect --output-manifest --profile sepolia

What are your thoughts around that?

[knownasred]

The output option is just to use existing standards, as it taps from the experience of kubectl users.

As you say, using the existing sozo inspect infrastructure seems like the best idea, I'm just worried that because it is so tightly linked to the existing build manifests that it may limit its potential uses.

In my opinion, the best course of action might be to use the inspect --output-manifest for a first version, and if it sees some adoption, maybe add a new sozo manifest that doesn't depend on having the project locally down the road if the need arises

[glihm]

The output option is just to use existing standards, as it taps from the experience of kubectl users.

As you say, using the existing sozo inspect infrastructure seems like the best idea, I'm just worried that because it is so tightly linked to the existing build manifests that it may limit its potential uses.

In my opinion, the best course of action might be to use the inspect --output-manifest for a first version, and if it sees some adoption, maybe add a new sozo manifest that doesn't depend on having the project locally down the road if the need arises

sozo inspect is also working even if nothing is present locally. Hence, as you suggested we may start with a sozo inspect --output-manifest for now, which will force sozo inspect to fetch from the chain instead of looking for information in the local one.

[knownasred]

sozo inspect is also working even if nothing is present locally.

It still requires the Scarb.toml and associated dojo_${profile}.toml files that makes it harder to generate a manifest from the frontend directory, for example.

But I agree that sozo inspect --output-manifest is a good idea. I will try to implement it next week, I'll keep you updated here :)

[glihm]

sozo inspect is also working even if nothing is present locally.

It still requires the Scarb.toml and associated dojo_${profile}.toml files that makes it harder to generate a manifest from the frontend directory, for example.

But I agree that sozo inspect --output-manifest is a good idea. I will try to implement it next week, I'll keep you updated here :)

Sounds good! Yes right, for now you still need the Scarb.toml and the profile in order to have sozo parsing the required information to correctly fetch the data.

We may see in the future how this command could be as standalone as possible to ease a front-end dev to just use rpc + world address to generate it. But may be challenging due to the data generated by the compilation that are not onchain.

[glihm]

Any update on what we could implement on this @knownasred? Wondering if with sozo inspect we have all the info, we should. But may be more calls to the chain to actually have all the details (like ABI, class hashes, etc..).

[knownasred]

gm! I need to get to actually implementing it (I'm focussing on our next project for now), but from by initial look, this might be more than enough to implement the sozo inspect --output-profile. A lot of the data is already provided:

So it seems like it's just a matter of exporting it in the good format, and possibly take some inspiration from what is being done in starkli: https://github.com/xJonathanLEI/starkli/blob/master/src/subcommands/class_at.rs#L37-L39

[glihm]

Not sure about the link with Starkli, we already have the class hash it is not displayed in the first inspect though.

No problem if you don't have the bandwidth but the idea is clear, I'll check to do it or someone from the team (or a contributor interested into it).

I need to get to actually implementing it (I'm focussing on our next project for now), but from by initial look, this might be more than enough to implement the sozo inspect --output-profile. A lot of the data is already provided:

You meant sozo inspect --output-manifest?

[knownasred]

Not sure about the link with Starkli, we already have the class hash it is not displayed in the first inspect though.

We have the class hash, but I was searching for a way to extract the ABI, and it seems like it is possible without too much hassle.

No problem if you don't have the bandwidth but the idea is clear, I'll check to do it or someone from the team (or a contributor interested into it).

I should have enough time to get working on it between this weekend and next week, I'll keep you updated on that

You meant sozo inspect --output-manifest?

Yeah, I should have checked before sending '^^

[glihm]

Ah ok, in a mode where everything is reconstructed from chain, that's true we will need to fetch those informations.

sounds good, keep me posted, and than you again for the will to contribute on that!