Can't work with > 2GiB memory due to ptr being interpreted as i32

[alecmocatta]

I get this error when allocating > 2GiB and then calling into JS:

Failed to execute 'decode' on 'TextDecoder': The encoded data was not valid.
    at getStringFromWasm0 (fu0min2hevxxsl6tfui2k8dfe:44)
    at imports.wbg.__wbg_setnodeValue_4a75b94edda71829 (fu0min2hevxxsl6tfui2k8dfe:360)

It's due to the ptr passed to getStringFromWasm0 being negative. If I add 2^32 to it (or do ptr >>> 0), then it is correct. I assume at some point it's being mistakenly interpreted as i32.

Here's the test case I used to narrow this down if it's useful:

use web_sys::{HtmlBodyElement, HtmlDocument};

#[wasm_bindgen(start)]
pub fn main() -> Result<(), JsValue> {
    let window = web_sys::window().unwrap();
    let document: HtmlDocument = window.document().unwrap().dyn_into().unwrap();
    let body: HtmlBodyElement = document.body().unwrap().dyn_into().unwrap();
    let x = document.create_text_node("abc");
    body.append_child(&x).unwrap();
    for i in 1_000_000..2_000_000 {
        let y = (0..(512 * 1024 * 1024))
            .map(|y| y as u8)
            .collect::<Vec<u8>>();
        std::mem::forget(y);
        let s = format!(
            "{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}",
            i, i, i, i, i, i, i, i, i, i, i, i, i, i, i, i, i, i, i, i
        );
        x.set_node_value(Some(&s));
        std::mem::forget(s);
    }
    Ok(())
}

[alexcrichton]

Thanks for the report! This is unfortunately probably an issue with most of the bindings we have, AFAIK we don't do >>>0 for anything internal right now, only when passsing unsigned integers out to JS. Definitely a bug we should fix though! I think this would probably amount to changing bits and pieces in this file, and ideally we could find a relatively deep place to flag pointers and lengths as u32 instead of i32 to fix most of these issues all at once.

[Dollab]

We are definitely hitting this wall on some of the largest file we use in our wasm application. Is there a current plan to resolve this and what could we do to help ?

[codingstar]

We are definitely hitting this wall on some of the largest file we use in our wasm application. Is there a current plan to resolve this and what could we do to help ?

+1