use wencoder for dummy instead of wat

[misapprehand]

1. extract wat generator to other file
2. add wencoder generator
3. splite wat genrator and wencoder to different function in dummy.rs

[misapprehand]

fix #2498

[github-actions[bot]]

Subscribe to Label Action

cc @fitzgen

This issue or pull request has been labeled: "fuzzing"

Thus the following users have been cc'd because of the following labels:

• fitzgen: fuzzing

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[misapprehand]

@fitzgen module_section is only in module link proposals and the section order is different from current order. I use the current one and add module_section to the data section. If I do not that, module_section is useless. So I not sure if it is right? Thanks

    pub fn finish(self) -> Vec<u8> {
        let mut module = Module::new();

        ....
        module.section(&self.module_section);   <---  module_section

        module.finish()
    }

[fitzgen]

Yeah, our fuzzers support module linking, and we don't want to lose that coverage, so you'll have to maintain that functionality as well. That means that there are some sections that can be repeated in any order at the start of a module:

This proposal instead loosens the section ordering rules such that the 5 initial sections (Type, Import, Module, Instance, Alias) can appear in any order, any number of times each.

If you're having trouble adding these sections in the right order, it may make sense to have a initial_sections: Vec<InitialSection> member in the generator, where InitialSection is an enum of those 5 initial sections, and whenever you need to add something to an initial section, if the last section in self.initial_sections isn't the section you need to add to, then push a new section of the right type and add your entity to that new section. Does that make sense?

Again, if this is going too deep and you want to back out, no worries, I understand that this wasn't really the "good first issue" I thought it was, sorry :(

[fitzgen]

By the way, I've added some tests for the dummy imports behavior to the main branch, and they might be helpful for your work in this PR since they check the expected results of the dummy imports (without requiring all the rest of the fuzzing infrastructure). If you rebase on main then you can run them via

cargo test -p wasmtime-fuzzing

[misapprehand]

This proposal instead loosens the section ordering rules such that the 5 initial sections (Type, Import, Module, Instance, Alias) can appear in any order, any number of times each.

@fitzgen I try to catch your point

1. We use proposal section order
2. ModuleSection can put any place in the module

Next is my solution for section order. I set the module_section at last. Is it right?

    pub fn finish(self) -> Vec<u8> {
        let mut module = Module::new();

        module.section(&self.custom_section);
        module.section(&self.type_section);
        module.section(&self.import_section);
        module.section(&self.function_section);
        module.section(&self.table_section);
        module.section(&self.memory_section);
        module.section(&self.global_section);
        module.section(&self.export_section);
        module.section(&self.start_section);
        module.section(&self.element_section);
        module.section(&self.code_section);
        module.section(&self.module_section);

        module.finish()
    }

[misapprehand]

@fitzgen I pass the cargo test -p wasmtime-fuzzing, but can not pass ci test. And I also pass the

cargo test \
>       --features test-programs/test_programs \
>       --all \
>       --exclude lightbeam \
>       --exclude wasmtime-lightbeam \
>       --exclude wasmtime-wasi-nn \
>       --exclude wasmtime-wasi-crypto \
>       --exclude peepmatic \
>       --exclude peepmatic-automata \
>       --exclude peepmatic-fuzzing \
>       --exclude peepmatic-macro \
>       --exclude peepmatic-runtime \
>       --exclude peepmatic-test \
>       --exclude peepmatic-souper

which broken in ci.

And when i run ./ci/run-experimental-x64-ci.sh, it fails

 cargo:rerun-if-changed=wasi-tests/src/lib.rs

  --- stderr
     Compiling wasi v0.10.0+wasi-snapshot-preview1
  error[E0463]: can't find crate for `core`
    |
    = note: the `wasm32-wasi` target may not be installed

  error: aborting due to previous error

  For more information about this error, try `rustc --explain E0463`.
  error: could not compile `wasi`

  To learn more, run the command again with --verbose.
  thread 'main' panicked at 'Building tests failed: exit code: 101', crates/test-programs/build.rs:69:13
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/c0b64d97beebb09325b5587abed39f4f1621026f/library/std/src/panicking.rs:493:5
     1: std::panicking::begin_panic_fmt
               at /rustc/c0b64d97beebb09325b5587abed39f4f1621026f/library/std/src/panicking.rs:435:5
     2: build_script_build::wasi_tests::build_tests
               at ./build.rs:69:13
     3: build_script_build::wasi_tests::build_and_generate_tests
               at ./build.rs:49:9
     4: build_script_build::main
               at ./build.rs:8:5
     5: core::ops::function::FnOnce::call_once
               at /rustc/c0b64d97beebb09325b5587abed39f4f1621026f/library/core/src/ops/function.rs:227:5

the result of rustup target list is. I have already install wasm32-wasi

thumbv7neon-unknown-linux-gnueabihf
thumbv8m.base-none-eabi
thumbv8m.main-none-eabi
thumbv8m.main-none-eabihf
wasm32-unknown-emscripten
wasm32-unknown-unknown (installed)
wasm32-wasi (installed)                                <----- installed
x86_64-apple-darwin
x86_64-apple-ios
x86_64-fortanix-unknown-sgx
x86_64-fuchsia
x86_64-linux-android
x86_64-pc-windows-gnu
x86_64-pc-windows-msvc
x86_64-rumprun-netbsd
x86_64-sun-solaris
x86_64-unknown-freebsd
x86_64-unknown-illumos
x86_64-unknown-linux-gnu (installed)
x86_64-unknown-linux-gnux32
x86_64-unknown-linux-musl
x86_64-unknown-netbsd

Can you help me ? Thanks

[cfallin]

@misapprehand the run-experimental-x64-ci.sh script uses nightly Rust (see in the script the $CARGO_VERSION variable), so you'll need to install the wasm32-wasi target for nightly as well:

rustup target add --toolchain nightly wasm32-wasi

[fitzgen]

@misapprehand if you want to run all tests for all crates, you need to make sure you have git submodules checked out and, as @cfallin mentioned, the wasm32-wasi target installed. Run thse commands before trying again:

rustup target add wasm32-wasi
git submodule update --init

Next is my solution for section order. I set the module_section at last. Is it right?

Not quite.

The order is

• (Type, Import, Module, Instance, Alias) sections repeated in any order, any number of times. (This is where the initial_sections vec comes in that I mentioned above)
• Function section
• Table section
• Memory section
• Global section
• Export section
• Start section
• Element section
• Code section
• Data section

Where custom sections can come in between all other sections. Although we shouldn't need any of the custom, start, element, or data sections when generating these dummy imports.

[fitzgen]

I pass the cargo test -p wasmtime-fuzzing

Did you rebase on top of the main branch? Because I wouldn't expect the current iteration of code to pass the tests on the main branch.

[misapprehand]

I pass the cargo test -p wasmtime-fuzzing

Did you rebase on top of the main branch? Because I wouldn't expect the current iteration of code to pass the tests on the main branch.

Sorry, I forget it. Now, I can run test. With your test program, I think I see the target.

[alexcrichton]

This is pretty out of date now at this point so I'm going to close this. I think the change would still be good to have though!