msw Request bodies of URLSearchParams can not be read

Prerequisites

[X] I confirm my issue is not in the opened issues
[X] I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

[X] I'm using the latest msw version
[X] I'm using Node.js version 14 or higher

Node.js version

v16.13.0

Reproduction repository

https://github.com/markdon/cra-msw

Reproduction steps

clone repo npm i DEBUG=* npm run test -- --watchAll=false

Current behavior

I think this is related to https://github.com/mswjs/msw/issues/1318

When handling a fetch request where the request body is a URLSearchParameters object, the body can not be read in the handler. This was previously available through request.body (v0.27.1). request.body is now deprecated.

I would expect to be able to just use the newer request.text(), however this seems to throw an error parsing the body.

Debug logs. See ERR_INVALID_ARG_TYPE

➜ cra-msw git:(main) DEBUG=* npm run test -- --watchAll=false

[email protected] test react-scripts test "--watchAll=false"

babel:config:loading:files:plugins Loaded preset '/Users/mark/projects/cra-msw/node_modules/babel-preset-react-app/index.js' from '/Users/mark/projects/cra-msw'. +0ms babel:config:loading:files:plugins Loaded preset '/Users/mark/projects/cra-msw/node_modules/babel-preset-jest/index.js' from '/Users/mark/projects/cra-msw'. +2ms http constructing the interceptor... +0ms xhr constructing the interceptor... +0ms setup-server constructing the interceptor... +0ms http:on adding "request" event listener: setupServerListener +0ms async-event-emitter:on adding "request" listener... +0ms xhr:on adding "request" event listener: setupServerListener +0ms async-event-emitter:on adding "request" listener... +0ms http:on adding "response" event listener: +0ms async-event-emitter:on adding "response" listener... +0ms xhr:on adding "response" event listener: +0ms async-event-emitter:on adding "response" listener... +0ms setup-server:apply applying the interceptor... +0ms async-event-emitter:activate set state to: ACTIVE +0ms setup-server:apply activated the emiter! ACTIVE +0ms setup-server retrieved global instance: undefined +4ms setup-server:apply no running instance found, setting up a new instance... +1ms setup-server:setup applying all 2 interceptors... +0ms setup-server:setup applying "ClientRequestInterceptor" interceptor... +0ms http:apply applying the interceptor... +0ms async-event-emitter:activate set state to: ACTIVE +0ms http:apply activated the emiter! ACTIVE +0ms http retrieved global instance: undefined +4ms http:apply no running instance found, setting up a new instance... +0ms http:setup native "http" module patched! +0ms http:setup native "https" module patched! +0ms http set global instance! http +0ms setup-server:setup adding interceptor dispose subscription +1ms setup-server:setup applying "XMLHttpRequestInterceptor" interceptor... +0ms xhr:apply applying the interceptor... +0ms async-event-emitter:activate set state to: ACTIVE +0ms xhr:apply activated the emiter! ACTIVE +0ms xhr retrieved global instance: undefined +5ms xhr:apply no running instance found, setting up a new instance... +0ms xhr:setup patching "XMLHttpRequest" module... +0ms xhr:setup native "XMLHttpRequest" module patched! XMLHttpRequestOverride +0ms xhr set global instance! xhr +0ms setup-server:setup adding interceptor dispose subscription +0ms setup-server set global instance! setup-server +1ms xhr:request POST /login open { method: 'POST', url: '/login', async: true, user: undefined, password: undefined } +0ms xhr:request POST /login reset +1ms xhr:request POST /login readyState change 0 -> 1 +1ms xhr:request POST /login triggering readystate change... +0ms xhr:request POST /login trigger "readystatechange" (1) +0ms xhr:request POST /login resolve listener for event "readystatechange" +0ms xhr:request POST /login set request header "content-type" to "application/x-www-form-urlencoded;charset=UTF-8" +0ms xhr:request POST /login send POST /login +0ms xhr:request POST /login request headers HeadersPolyfill { [Symbol(normalizedHeaders)]: { 'content-type': 'application/x-www-form-urlencoded;charset=UTF-8' }, [Symbol(rawHeaderNames)]: Map(1) { 'content-type' => 'content-type' } } +1ms xhr:request POST /login emitting the "request" event for 1 listener(s)... +0ms async-event-emitter:emit emitting "request" event... +0ms async-event-emitter:openListenerQueue opening "request" listeners queue... +0ms async-event-emitter:openListenerQueue no queue found, creating one... +0ms async-event-emitter:emit appending a one-time cleanup "request" listener... +0ms async-event-emitter:openListenerQueue opening "request" listeners queue... +0ms async-event-emitter:openListenerQueue returning an exising queue: [] +0ms async-event-emitter:on awaiting the "request" listener... +11ms xhr:request POST /login awaiting mocked response... +2ms async-event-emitter:on "TypeError [ERR_INVALID_ARG_TYPE]: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams" listener has rejected! +13ms xhr:request POST /login middleware function threw an exception! TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams at new NodeError (node:internal/errors:371:5) at TextDecoder.decode (node:internal/encoding:413:15) at Object.decodeBuffer (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18) at RestRequest. (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/IsomorphicRequest.ts:59:12) at step (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:33:23) at Object.next (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:14:53) at /Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:8:71 at new Promise () at Object..__awaiter (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:4:12) at RestRequest.Object..IsomorphicRequest.text (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/lib/IsomorphicRequest.js:73:16) { code: 'ERR_INVALID_ARG_TYPE' } +12ms xhr:request POST /login trigger "error" (1) +8ms xhr:request POST /login resolve listener for event "error" +1ms xhr:request POST /login abort +0ms xhr:request POST /login readyState change 1 -> 0 +0ms xhr:request POST /login trigger "abort" (0) +0ms xhr:request POST /login resolve listener for event "abort" +0ms async-event-emitter:emit cleaned up "request" listeners queue! +27ms setup-server:dispose disposing the interceptor... +0ms setup-server retrieved global instance: BatchInterceptor +49ms setup-server cleared global instance! setup-server +0ms setup-server:dispose global symbol deleted: undefined +0ms setup-server:dispose disposing of 2 subscriptions... +0ms http:dispose disposing the interceptor... +0ms http retrieved global instance: ClientRequestInterceptor +51ms http cleared global instance! http +0ms http:dispose global symbol deleted: undefined +0ms http:dispose disposing of 2 subscriptions... +0ms http:setup native "http" module restored! +51ms http:setup native "https" module restored! +0ms http:dispose disposed of all subscriptions! 0 +0ms async-event-emitter:deactivate removing all listeners... +0ms async-event-emitter:removeAllListeners event: undefined +0ms async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms async-event-emitter:deactivate set state to: DEACTIVATED +0ms http:dispose destroyed the listener! +0ms xhr:dispose disposing the interceptor... +0ms xhr retrieved global instance: XMLHttpRequestInterceptor +50ms xhr cleared global instance! xhr +0ms xhr:dispose global symbol deleted: undefined +0ms xhr:dispose disposing of 1 subscriptions... +0ms xhr:setup native "XMLHttpRequest" module restored! XMLHttpRequest +51ms xhr:dispose disposed of all subscriptions! 0 +1ms async-event-emitter:deactivate removing all listeners... +0ms async-event-emitter:removeAllListeners event: undefined +0ms async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms async-event-emitter:deactivate set state to: DEACTIVATED +0ms xhr:dispose destroyed the listener! +0ms setup-server:dispose disposed of all subscriptions! 0 +2ms async-event-emitter:deactivate removing all listeners... +0ms async-event-emitter:removeAllListeners event: undefined +0ms async-event-emitter:removeAllListeners cleared the listeners queue! Map(0) {} +0ms async-event-emitter:deactivate set state to: DEACTIVATED +0ms setup-server:dispose destroyed the listener! +0ms FAIL src/Login.test.ts fetch ✕ can login (35 ms)

● fetch › can login

expect(received).resolves.toBeInstanceOf()

Received promise rejected instead of resolved
Rejected to value: [TypeError: Network request failed]

  2 | describe('fetch', ()=>{
  3 |   it('can login', async ()=>{
> 4 |     await expect(fetch('/login', {
    |           ^
  5 |       method: 'POST',
  6 |       body: new URLSearchParams('username=admin&password=admin')
  7 |     })).resolves.toBeInstanceOf(Response);

  at expect (node_modules/expect/build/index.js:178:15)
  at Object.<anonymous> (src/Login.test.ts:4:11)
  at TestScheduler.scheduleTests (node_modules/@jest/core/build/TestScheduler.js:333:13)
  at runJest (node_modules/@jest/core/build/runJest.js:404:19)
  at _run10000 (node_modules/@jest/core/build/cli/index.js:320:7)
  at runCLI (node_modules/@jest/core/build/cli/index.js:173:3)

Test Suites: 1 failed, 1 total Tests: 1 failed, 1 total Snapshots: 0 total Time: 0.834 s, estimated 1 s Ran all test suites.

Expected behavior

A request body of type URLSearchParameters should be able to be read in a handler. I suggest via request.text(), which would return the value of URLSearchParameters.toString().

Aug 17 '22 07:08 markdon

Hey, @markdon. Thanks for reporting this.

We are working on improving how request bodies are read, and this use case is going to the list of things we need to account for. Meanwhile, you can still use req.body if that used to work previously. The logic there should remain the same.

Aug 17 '22 11:08 kettanaito

Actually, I only spent the time investigating this after req.body.toString() started throwing a very similar error :( Just accessing req.body at all throws the same error.

I can refactor the few of my tests this affected to move to latest msw.

Debug log snippet

xhr:request POST /login middleware function threw an exception! TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams
    at new NodeError (node:internal/errors:371:5)
    at TextDecoder.decode (node:internal/encoding:413:15)
    at decodeBuffer (/Users/mark/projects/cra-msw/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18)
    at RestRequest.body (/Users/mark/projects/cra-msw/node_modules/msw/src/utils/request/MockedRequest.ts:111:18)
    at /Users/mark/projects/cra-msw/src/setupTests.ts:8:44
    at /Users/mark/projects/cra-msw/node_modules/msw/src/handlers/RequestHandler.ts:232:55
    at RestHandler.run (/Users/mark/projects/cra-msw/node_modules/msw/src/handlers/RequestHandler.ts:215:34)
    at /Users/mark/projects/cra-msw/node_modules/msw/src/utils/getResponse.ts:50:34
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at getResponse (/Users/mark/projects/cra-msw/node_modules/msw/src/utils/getResponse.ts:41:18) {
  code: 'ERR_INVALID_ARG_TYPE'
} +12ms

Aug 17 '22 22:08 markdon

Oh, so the reason is not to read the body but to construct it. He'd have to support that.

We're using something called IsomorphicRequest abstract that allows us to handle different request body types in the same manner. That abstraction only accepts ArrayBuffer as request body (which we want to keep) but the consumer (MSW in this case) can transform the body however they need to eventually provide the buffer.

So, I'd imagine this support to be:

Check when the body is URLSearchParams.
Serialize that somehow.
Turn it into an ArrayBuffer.
Make sure it's de-serializable as well.

Contributions are welcome!

Aug 19 '22 10:08 kettanaito

Sounds pretty straight forward to implement! I would love to contribute, if I hadn't just signed up to a project with a tight timeline for the next few months.

I will check back later!

Aug 21 '22 23:08 markdon

My workaround, which was actually for FormData, is the following:

jest.spyOn(global, "FormData").mockImplementation(mockFormDataForMSW);
jest.mock("util", mockNodeUtilForMSW);

function mockFormDataForMSW() {
  return new FormDataFakeArrayBuffer(0) as unknown as FormData;
}

function mockNodeUtilForMSW() {
  const real = jest.requireActual("util") as typeof import("util");
  return {
    ...real,
    TextDecoder: class extends real.TextDecoder {
      decode(
        input?: NodeJS.ArrayBufferView | ArrayBuffer | null,
        options?: { stream?: boolean | undefined },
      ): string {
        if (input instanceof FormDataFakeArrayBuffer) return input.toString();

        return super.decode(input, options);
      }
    },
  };
}

class FormDataFakeArrayBuffer extends ArrayBuffer {
  private params = new URLSearchParams();

  append(param: string, value: string) {
    return this.params.append(param, value);
  }

  toString() {
    return this.params.toString();
  }
}

For reference/SEO, the error was:

The input argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData

or

The input argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams

Sep 02 '22 08:09 ianldgs

I'm not sure why I didn't consider this before but surely the easiest workaround (for URLSearchParams) is to create the request with a string instead of URLSearchParams object. e.g. use URLSearchParams.toString() for the body when calling fetch.

The MSW handler will always need to parse the text body with new URLSearchParams(await body.text()) as there is no Request method that gets the body as a URLSearchParams object.

Sep 04 '22 23:09 markdon

Same issue as mine in #1327 but with FormData bodies. msw just plain broke reading request bodies in v0.44.0.

New features keep getting added in the 0.x line so that it's totally OK to break stuff from a semver perspective. And bugs are not fixed.

#sadness

Sep 09 '22 18:09 ddolcimascolo

Same problem here.

Oct 07 '22 13:10 chawax

I used Axios instead of Fetch and it works !

Oct 07 '22 13:10 chawax

@ddolcimascolo, a true sadness is having a 7M/week downloaded library used by FAANG and having to develop it over weekends instead of spending time with one's family (which I've promised myself not to do anymore, thus you will have issues that I don't have time/desire to work on). But hey, you can take a step towards resolving that sadness, here you go.

This is going to be fixed by #1436 so I'm not going to do any work around this until that API lands. If you're blocked by this, consider contributing, I'd do my best to review your code and release it.

Nov 18 '22 21:11 kettanaito

Yeah. My message above was itself sad, sorry about that, probably written while closing yet another upgrade merge request on MSW that I can't merge because of this blocker. You do a great job in maintaining OSS projects, keep up the great work!

Since my message above I've contributed a fix, even though you implemented #1436 in the end which is a really nice addition.

Cheers, David

Nov 19 '22 08:11 ddolcimascolo

I will check back later!

It looks like a lot of progress has been made towards the Fetch API update that replaces the related code and fixes this issue. There's a beta available to try.

I had a quick look at making a fix for the current release of msw but @mswjs/interceptors has already moved on from the related IsomorphicRequest class and the Fetch API update looks pretty close to done anyway.

Nov 28 '22 00:11 markdon

Hey, just little question from me. I am using latest msw, node and react-query@^3.39.2. And I am trying to connect with a Keycloak (newest stable version) for fresh token before running tests. I have to use URLSearchParams formatting in body to construct fetch and I am getting this mentioned error from msw:

"The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of URLSearchParams."

I guess, #1436 is fixing it also for this case, right? Otherwise what is the status of completition @kettanaito ?

Jan 25 '23 11:01 DusanPausly

https://stackoverflow.com/questions/75535744/cant-get-body-using-msw-and-redux-toolkit/75543115#75543115

Feb 23 '23 09:02 erperejildo

@DusanPausly, correct. The current version of MSW doesn't support URLSearchParams as the request body. The Fetch API feature will add that support.

Feb 23 '23 13:02 kettanaito

Released: v2.0.0 🎉

This has been released in v2.0.0!

📄 Release notes
📦 npm package

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.

Predictable release automation by @ossjs/release.

Oct 23 '23 08:10 kettanaito

msw msw copied to clipboard

Request bodies of URLSearchParams can not be read

Prerequisites

Environment check

Node.js version

Reproduction repository

Reproduction steps

Current behavior

Expected behavior

Released: v2.0.0 🎉

msw
msw copied to clipboard