Response::new_with_opt_u8_array_and_init unexpected array contents

[forficate]

Summary

I've raised this as a question as not sure if it's a Cloudflare worker issue or WASM issue on my side or WASM web_sys bug.

I seem to have issues with web_sys::Response::new_with_opt_u8_array_and_init returning unexpected output.

Take the below function:

pub fn response_as_bytes(s: String) -> Result<Response, JsValue> {
    let mut init = ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/html; charset=utf-8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);

    let mut body = s.into_bytes();

    web_sys::Response::new_with_opt_u8_array_and_init(Some(&mut body), &init)
}

I expect the function called with Hello World! to return a response with an ArrayBuffer containing the utf8 string Hello World!.

The output of this response is corrupt in that it does not contain a utf8 string Hello World!. I get the characters nn displayed in a browser.

If I debug the response from Javascript I get the below:

const resp = await execute(request)
const body = await resp.arrayBuffer()
console.log(body)
    
[[Uint8Array]]: Uint8Array(12)

    0: 16
    1: 109
    2: 19
    3: 0
    4: 16
    5: 109
    6: 19
    7: 0
    8: 32
    9: 0
    10: 0
    11: 0  

The array looks to be of the correct size, 12, but the contents are not correct.

The below is the expected contents:

const encoder = new TextEncoder()
const view = encoder.encode('Hello World!')
console.log(view)

int8Array(12) [72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33]

To verify I am correctly encoding the string I created the below function:

pub fn response_as_bytes_debug(s: String) -> Result<Response, JsValue> {
    let mut init = ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/html; charset=utf-8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);

    let mut body = s.into_bytes();
    let body = serde_json::to_string(&body).unwrap();

    web_sys::Response::new_with_opt_str_and_init(Some(&body), &init)
}

When I run this function I get the below response, which is what I expect matching the contents of new TextEncoder().encode('Hello World!').

[72,101,108,108,111,32,119,111,114,108,100,33]

Looking Response::new_with_opt_u8_array_and_init calls the js Response object with a ArrayBuffer constructor argument?

The below looks to work in Javascript:

const encoder = new TextEncoder()
const view = encoder.encode('Hello World')
const resp = new Response(view)
resp.text().then(console.log)

> Hello World

The following repo contains a full poc test case https://github.com/forficate/cloudflare_wasm_bug_testcase With the above I'm not sure if I'm misunderstanding Response::new_with_opt_u8_array_and_init or if there's a bug with the bytes being passed through not matching the bytes received on the js end.

I've only started looking in to WASM so not quite sure where next to go debugging.

[forficate]

I've done some more debugging.

In my Javascript entrypoint I have adjusted it to the below adding a custom my_response shim method:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

const { execute } = wasm_bindgen;
const instance = wasm_bindgen(wasm)

function my_response(body, init) {
  return new Response(body, init)
}

async function handleRequest(request) {
    await instance;
    const resp = await execute(request)
    return resp
}

On my WASM side I have the two functions:

// 1) Use the built in web_sys::Response::new_with_opt_u8_array_and_init func
pub fn response_as_bytes(s: String) -> Result<Response, JsValue> {
    let mut init = ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/html; charset=utf-8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);

    let mut body = s.into_bytes();

    web_sys::Response::new_with_opt_u8_array_and_init(Some(&mut body), &init)
}

// 2) Use my custom my_response shim
pub fn response_as_bytes_custom(s: String) -> Result<Response, JsValue> {
    let mut body = s.into_bytes();

    let mut init = web_sys::ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/plain; utf8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);
    init.status(200);

    Ok(my_response(&mut body, init))
}

And an extern declaration for my_response:

#[wasm_bindgen]
extern "C" {
  fn my_response(body: &mut [u8], init: web_sys::ResponseInit) -> web_sys::Response;
}

The below fails using the standard web_sys::Response. In the browser I see the characters 'HH'.

#[wasm_bindgen]
pub async fn execute(req: Request) -> Result<Response, JsValue> {
    response_as_bytes(String::from("Hello World!"))
}

Swapping the above out for my custom shim works as expected, I get the text "Hello World!" in the browser:

#[wasm_bindgen]
pub async fn execute(req: Request) -> Result<Response, JsValue> {
    response_as_bytes_custom(String::from("Hello World!"))
}

What is interesting though is when I refactored response_as_bytes_custom to match response_as_bytes for the last call.

Take the below which works as expected:

pub fn response_as_bytes_custom(s: String) -> Result<Response, JsValue> {
    let mut body = s.into_bytes();

    let mut init = web_sys::ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/plain; utf8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);
    init.status(200);

    Ok(my_response(&mut body, init))
}

I then move the let mut body = s.into_bytes() line to the bottom and the function breaks and I get garbled output in the browser. See below breaking change:

pub fn response_as_bytes_custom(s: String) -> Result<Response, JsValue> {
    let mut init = web_sys::ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/plain; utf8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);
    init.status(200);

    let mut body = s.into_bytes();
    Ok(my_response(&mut body, init))
}

After seeing this I update the broken response_as_bytes moving let mut body = s.into_bytes() to the top of the function to see if that fixes it:

pub fn response_as_bytes(s: String) -> Result<Response, JsValue> {
    let mut body = s.into_bytes();

    let mut init = ResponseInit::new();
    let mut headers: HashMap<&str, &str> = HashMap::new();
    headers.insert("Content-Type", "text/html; charset=utf-8");
    headers.insert("Cache-Control", "no-cache");

    let headers = wasm_bindgen::JsValue::from_serde(&headers).unwrap();
    init.headers(&headers);

    web_sys::Response::new_with_opt_u8_array_and_init(Some(&mut body), &init)
}

And indeed it does fix it. I get the expected output in my browser "Hello World!"

Now I am slightly confused. The result of the function is dependent on the placement of let mut body = s.into_bytes();. If I get the bytes just before creating the Response the function fails i.e.:

 let mut body = s.into_bytes();
    web_sys::Response::new_with_opt_u8_array_and_init(Some(&mut body), &init)

If I call let mut body = s.into_bytes(); at the start of the function before creating the ResponseInit object it works completely fine.

[jonboj]

Many details, are Response an interface of the Fetch api ? https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

[NilsIrl]

I seem to be hitting a similar issue. I'm calling a javascript function with a &[u8] and it seems like the Uint8Array actually passed on the other side contains 8 bytes of garbage replacing the first 8 bytes.

[NilsIrl]

Using wee_alloc instead of the default allocator makes it so that only the first 4 bytes are changed.

[artem-tim]

Same issue here. Seems to be Cloudflare specific. With wee_alloc, first 4 bytes are mangled (ie. transformed into garbage), and without it sometimes last bytes are mangled. I recommend using new_with_opt_str_and_init instead for now, dealing with small performance overhead of transforming &[u8] into &str