msw icon indicating copy to clipboard operation
msw copied to clipboard

Support "req.formData()"

Open ddolcimascolo opened this issue 2 years ago • 16 comments

Prerequisites

Environment check

  • [X] I'm using the latest msw version
  • [X] I'm using Node.js version 14 or higher

Node.js version

16.13.0

Reproduction repository

https://github.com/ddolcimascolo/msw-issue-1327

Reproduction steps

Axios to send a POST request with a FormData body. Try to use req.body in the handler

Current behavior

TypeError: The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData
        at new NodeError (node:internal/errors:371:5)
        at TextDecoder.decode (node:internal/encoding:413:15)
        at decodeBuffer (/data/dev/msw-issue-formdata/node_modules/@mswjs/interceptors/src/utils/bufferUtils.ts:11:18)
        at RestRequest.get body [as body] (/data/dev/msw-issue-formdata/node_modules/msw/src/utils/request/MockedRequest.ts:117:18)

Expected behavior

Maybe req.formData() as per https://developer.mozilla.org/en-US/docs/Web/API/Request/formData would be the way to go?

ddolcimascolo avatar Jul 13 '22 13:07 ddolcimascolo

I am using a custom fetcher with codegen to handle FormData mutations and have never had much luck getting it to work in msw so have been intercepting the request at the point where my data is submitted in my tests like this:

// Intercept the upload
  server.use(
    rest.post(
      "http://localhost:8000/fleetportalapi/graphiql",
      (req, res, ctx) =>
        res(
          ctx.json({
            data: {
              fileUploadMileage: {
                __typename: "FileUploadMileageError",
                errorMessage: "Error parsing the CSV file.",
              },
            },
          })
        )
    )
  );

This is not perfect - ideally I would be able to use graphql.mutation - but it was working fine until I updated to 0.44.0 today where I now get the same error:

console.error
    The "input" argument must be an instance of ArrayBuffer or ArrayBufferView. Received an instance of FormData

      at node_modules/msw/src/handlers/GraphQLHandler.ts:155:26
      at tryCatch (node_modules/msw/src/utils/internal/tryCatch.ts:9:5)
      at GraphQLHandler.parse (node_modules/msw/src/handlers/GraphQLHandler.ts:153:12)
      at GraphQLHandler.test (node_modules/msw/src/handlers/RequestHandler.ts:167:12)
      at node_modules/msw/src/utils/getResponse.ts:31:20
          at Array.filter (<anonymous>)
      at getResponse (node_modules/msw/src/utils/getResponse.ts:30:37)

FWIW my custom fetch looks like this (mostly borrowed from graphql-request:

import { extractFiles, isExtractableFile } from "extract-files";

const createRequestBody = <TVariables>(
  query: string,
  variables?: TVariables
) => {
  const { files, clone } = extractFiles(
    { query, variables },
    "",
    isExtractableFile
  );

  if (files.size === 0) {
    return JSON.stringify({ query, variables });
  }

  const form = new FormData();

  form.append("operations", JSON.stringify(clone));

  const map: { [key: number]: string[] } = {};
  let i = 0;
  files.forEach((paths) => {
    map[++i] = paths;
  });
  form.append("map", JSON.stringify(map));

  i = 0;
  files.forEach((paths, file) => {
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    form.append(`${++i}`, file as any);
  });

  return form as FormData;
};

export const useFetchData = <TData, TVariables>(
  query: string,
  options?: RequestInit["headers"]
): ((variables?: TVariables) => Promise<TData>) => {
  return async (variables?: TVariables) => {
    const body = createRequestBody(query, variables);

    const res = await fetch(url, {
      method: "POST",
      headers: {
        ...(typeof body === "string"
          ? { "Content-Type": "application/json" }
          : {}),
        ...(options ?? {}),
      },
      body,
    });

    const json = await res.json();

    if (json.errors) {
      const { message } = json.errors[0] || "Error..";
      throw new Error(message);
    }

    return json.data;
  };
};

My issue would be solved if msw could handle these kind of requests. For now I'll roll back to 0.42.1.

robcaldecott avatar Jul 13 '22 15:07 robcaldecott

@ddolcimascolo Can you provide reproduction code for this error?

95th avatar Jul 16 '22 07:07 95th

@95th I didn't forget, but facing some other troubles atm. If you want to give it a try it's really just

  1. Send a request with axios using a FormData body to an URL mocked with MSW
  2. Access req.body in the request handler

Was working fine in the previous minor.

Cheers, David

ddolcimascolo avatar Jul 16 '22 15:07 ddolcimascolo

@ddolcimascolo I tried this but its working fine for me. any special data included in the FormData?

95th avatar Jul 16 '22 16:07 95th

Nothing special. But I realize I have not given many details about the context... Were doing Jest tests with jsdom, so the FormData is what jsdom provides... Maybe it's the culprit but this was running smoothly before.

I'll definitely work on reproducing this in a repo

ddolcimascolo avatar Jul 16 '22 18:07 ddolcimascolo

Having same issue, can't pull file off with req.body.get("file"). Worked in previous versions.

const file = new File(["test"], "test.bad");
const formData = new FormData();
formData.append("file", file);
  
fetch("/api/uploadFile", {
    body: formData,
    method: "POST",
  })
rest.post(
    "/api/uploadFile",
    (req: MockedRequest<FormData>, res, ctx) => {
      const file = req.body.get("file"); // fails here, hangs
      
      ...
    }
)

Works in 0.43.1, stops working in 0.44.0+

nathanhannig avatar Jul 20 '22 00:07 nathanhannig

Thx @nathanhannig. I didn't manage to create a full reproduction repo, yet.

ddolcimascolo avatar Jul 20 '22 06:07 ddolcimascolo

@95th @nathanhannig I finally created the reproduciton repo @ https://github.com/ddolcimascolo/msw-issue-1327

ddolcimascolo avatar Jul 27 '22 14:07 ddolcimascolo

Guys, any news?

ddolcimascolo avatar Aug 03 '22 06:08 ddolcimascolo

@ddolcimascolo, no news so far. Anybody is welcome to take the reproduction repo above and step through this body function to find out what happens now with FormData bodies. Debugging this is about 80% of solving it.

kettanaito avatar Aug 03 '22 11:08 kettanaito

on 0.44.2 ->

I can read FormData from req.body, however it's not a FormData object, but instead a json representation - so I grab the file via req.body.file rather than req.body.get('file')

Screen Shot 2022-08-03 at 6 04 33 PM

Request data

Screen Shot 2022-08-03 at 6 06 19 PM

text() => gives the value of text2 in that debugger json() => throws (since its not json)

At some point it does seem like the file gets corrupted in 'upload' but I haven't gotten into where/how that could be improved/changed

mattcosta7 avatar Aug 03 '22 22:08 mattcosta7

Can that corruption be related to #1158 somehow?

kettanaito avatar Aug 29 '22 09:08 kettanaito

Hi everyone, still no fix to this issue? We're stuck a few versions behind because of this... Can you advise if this would be doable by someone that never contributed to msw? Is this a good first issue?

Cheers, David

ddolcimascolo avatar Sep 04 '22 15:09 ddolcimascolo

I have been stuck at 0.43.1 because of this isse

req.body was broken after that version (even though it incorrectly says it was depreciated)

nathanhannig avatar Sep 13 '22 17:09 nathanhannig

Welcome aboard 🙄

ddolcimascolo avatar Sep 13 '22 17:09 ddolcimascolo

req.body was broken after that version (even though it incorrectly says it was depreciated)

Hence why we've released it as a breaking change. It is getting deprecated, but it also drops some of the request reading methods since we didn't have the capacity to implement all of them.

Can you advise if this would be doable by someone that never contributed to msw? Is this a good first issue?

I don't see why it wouldn't be. This is a scoped known change, and you have previous versions that supported this to verify your implementation against.

How can I contribute?

  1. A/B what's going wrong with a FormData request body (req.body) in the resolver between 0.43.0 and 0.47.0. The fix will depend on the root cause: what exactly goes off? It may be useful to see how we parsed the request body before.
  2. Add the FormData case handling to the .body method here. If applicable, we can also add MockedRequest.formData() method and reuse it in the deprecated .body property.

The main task here is to determine how we used to handle FormData request bodies in the past. I honestly don't recall. I don't think we used polyfills for that, it must've been something else. If we could bring that something else back into .body it'd be great.

Volunteers are certainly welcome. I will help with the code review so we could ship this feature in a timely manner.

kettanaito avatar Sep 16 '22 17:09 kettanaito

Hey guys, my tentative in https://github.com/mswjs/msw/pull/1422

ddolcimascolo avatar Oct 02 '22 17:10 ddolcimascolo

@kettanaito Did you get a chance to review the PR?

Cheers, David

ddolcimascolo avatar Oct 06 '22 18:10 ddolcimascolo

Hey, @ddolcimascolo. Yes, thank you so much for opening it. I've left a few comments and I'd love to know your thoughts on those.

Also, it's worth mentioning that #1404 change is going to introduce .formData() as well but I don't think it's going to happen that fast.

kettanaito avatar Oct 10 '22 13:10 kettanaito

@kettanaito Thx for the feedback. I dropped Axios in favor of a raw XHR request but I have no clue how to handle your comment about FormData not existing in Node...

ddolcimascolo avatar Oct 10 '22 16:10 ddolcimascolo

I guess I stay on 0.38.1 until you implement the body getter...

boryscastor avatar Apr 20 '23 21:04 boryscastor

@boryscastor, the body getter is not coming back. Instead, in the next major release you will be able to read the request's body as form data like you would do in regular JavaScript:

rest.post('/login', async ({ request }) => {
  const data = await request.formData()
  const email = data.get('email')
})

When is this coming out? Sometime this year, hopefully.

When can I try this? You can try this right now. More details in #1464.

kettanaito avatar Apr 21 '23 18:04 kettanaito

what's the workaround until req.formData() exists? Can I do something manually to get the data from req.arrayBuffer()?

    const arrayBuffer = await req.arrayBuffer();
    const formData = undefined; // ??;

fibonacid avatar Aug 10 '23 16:08 fibonacid

@fibonacid, the FormData request body is actually a string in a predefined format:

const formData = new FormData()
formData.set('foo', 'bar')

const request = new Request('/url', { method: 'PUT', body: formData })
await request.text()
'------WebKitFormBoundaryY2WwRDBYmRbAKyLB\r\nContent-Disposition: form-data; name="foo"\r\n\r\nbar\r\n------WebKitFormBoundaryY2WwRDBYmRbAKyLB--\r\n'

You can read the request body as text to obtain that string. Then, you can feed it to any parser that would turn it back into a FormData instance. Unfortunately, the standard FormData constructor doesn't support that string as an input. You'd have to use third-party packages to do the job here.

You can use parse-multipart-data to parse that string into an object and then construct a FormData instance using that object.

Alternatively, consider switching to msw@next that ships with FormData built-in. More on that version here: #1464.

kettanaito avatar Aug 11 '23 10:08 kettanaito

@kettanaito thanks, this is very helpful!

fibonacid avatar Aug 11 '23 10:08 fibonacid

Came across this thread after facing 2 issues with MSW - all my graphql handlers throwing input instance of arraybuffer whenever i used formdata, and NetworkError when trying to access a body that was formdata. Will migrate to @next and give that a go. Thanks for the detailed migration guide, that'll save us a lot of time. Will edit this if it solves all of my problems

tomdglenn91 avatar Sep 11 '23 09:09 tomdglenn91

Released: v2.0.0 🎉

This has been released in v2.0.0!

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.


Predictable release automation by @ossjs/release.

kettanaito avatar Oct 23 '23 08:10 kettanaito