calamine icon indicating copy to clipboard operation
calamine copied to clipboard

Weird performance difference in wasm32 target

Open KivalM opened this issue 3 years ago • 1 comments

Hey. So I am currently working with calamine in a rust webassembly environment and I am experiencing an insane difference in performance compared to a unix environment. At the moment I have this bit of code

 let cursor = Cursor::new(buff.to_vec());
 let mut excel: Xlsx<_> = Xlsx::new(cursor).unwrap();
 let worksheets = excel.worksheets();

    for (name, range) in worksheetnames {
        let mut rows_vec = vec![vec![]; range.width()];

        for (i, row) in range.rows().take(10000).enumerate() {
            rows_vec.push(row.iter().map(|f| f.to_string()).collect::<Vec<String>>());
        }
    }

Where buff is just a vec of the bytes from the 60mb file. It is nothing too complicated, however the speed difference is surprising.

On a unix target the previous code runs in only 9112ms However on a WebAssembly target it runs in 33453ms.

Both are built for release.

Is there any reason why this would be the case?

KivalM avatar Aug 23 '22 14:08 KivalM

Just a small note. The line let mut excel: Xlsx<_> = Xlsx::new(cursor).unwrap(); seems to run quite fast. The slowdown is happening here.

excel.worksheets();

It seems that https://github.com/tafia/calamine/issues/258 this user's PoC seems to have a similar issue with loading 60MB files. But why is it significantly slower in wasm?

EDIT: Updated the code that is slow

KivalM avatar Aug 28 '22 13:08 KivalM

On a unix target the previous code runs in only 9112ms However on a WebAssembly target it runs in 33453ms.

Both are built for release.

Is there any reason why this would be the case?

In my opinion, that comparison is unfair to WASM. After all, WebAssembly is not native code:

WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine.

Just by running in a VM, one would expect some overhead and slowdown, compared to running a native binary. How much slower WASM is compared to native code can vary, depending on the code that is run. An example for this is described in the paper Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code by Abhinav Jangda, Bobby Powers, Emery D. Berger, and Arjun Guha from the University of Massachusetts Amherst. While this paper does not focus on Rust and your numbers may differ, it describes the slowdown as follows:

Across the SPEC CPU suite of benchmarks, we find a substantial performance gap: applications compiled to WebAssembly run slower by an average of 45% (Firefox) to 55% (Chrome), with peak slowdowns of 2.08× (Firefox) and 2.5× (Chrome). We identify the causes of this performance degradation, some of which are due to missing optimizations and code generation issues, while others are inherent to the WebAssembly platform.

(Bold text highlighted by me, it was not highlighted in the original.)

So I gather from this, that approx. 50 % slower execution is to be expected with the benchmark suite that was run for that paper, and I would expect similar slowdowns for other applications, too.

striezel avatar Oct 10 '22 11:10 striezel

I am seeing lot ofprojects with ~50% drop in performance due to wasm so I don't think this issue is specific to calamine, Closing it for now, feel free to reopen if you think otherwise.

tafia avatar Jun 25 '23 10:06 tafia