fluvio icon indicating copy to clipboard operation
fluvio copied to clipboard

[Bug]: Resource restrictions for WASM code execution in SmartStream

Open avinassh opened this issue 3 years ago • 2 comments

I was checking what all resource restrictions have been set for WASM code execution. All the tests have been done locally with the latest master 2c0005d

I created a smart stream filter which never really exits:

#[smartstream(filter)]
pub fn filter(record: &Record) -> bool {
    let str_result = std::str::from_utf8(record.value.as_ref());
    let string = match str_result {
        Ok(s) => s,
        _ => return false,
    };
    let mut c = 0;
    loop {
        c += 1;
    }
    eprintln!("{}", c);
    string.contains('a')
}

I ran the consumer on my terminal which just hung till I cancelled. The CPU usage shot to 70% (from 30%). I kept it running for few minutes, assuming if there were any max time set on these operations. Then I manually cancelled.

Also tried code with infinite recursion, this also behaves like previous code and CPU usage increased:

   // omitted for brevity

    // never ending recursion
    fn fact(x: u32) -> u32 { if x == 0 {1} else {fact(x)} }
    eprintln!("{:?}", fact(5));
    string.contains('a')
}

From our discussion from Discord, @sehz suggested setting consume_fuel to solve these issues.


Reporting for the sake of completeness, I am mentioning other things I tried. In the following, WASM engine gave error and consumer returned immediately:

  1. code with sleep statement
  2. code with process.exit

I wanted to test for memory limits, but could not come up with rust code which grew in memory. Related issues in wastime (#1872, #2273)

my code:

#[smartstream(filter)]
pub fn filter(record: &Record) -> bool {
    let str_result = std::str::from_utf8(record.value.as_ref());
    let string = match str_result {
        Ok(s) => s,
        _ => return false,
    };
    let mut my_vector = Vec::new();
    let random_string = String::from("abcdefghijklmnopqrstuvwxyz");
    loop {
        my_vector.push(random_string.clone());
    }
    my_vector.last().unwrap().contains('a')
}

But this was handled properly by the SPU with the following error:

Jul 15 19:49:00.148 ERROR stream fetch{starting_offset=0 replica=echo-0 sink=15}: fluvio_spu::services::public::stream_fetch: error: IoError(
    Custom {
        kind: Other,
        error: "filter err wasm trap: unreachable\nwasm backtrace:\n    0: 0x67a1 - <unknown>!alloc::raw_vec::RawVec<T,A>::reserve::do_reserve_and_handle::h7d6e15d165e7e76e\n    1: 0x9a1a - <unknown>!filter\nnote: run with `WASMTIME_BACKTRACE_DETAILS=1` environment variable to display more information\n",
    },
)

avinassh avatar Jul 15 '21 14:07 avinassh

Stale issue message

github-actions[bot] avatar Sep 14 '21 11:09 github-actions[bot]

Stale issue message

github-actions[bot] avatar Mar 24 '22 11:03 github-actions[bot]