Listening to <client>.rest response Event breaks <guild>.bans.fetch() and <guild>.invites.fetch()

[Larsundso]

Which package is this bug report for?

discord.js

Issue description

1. Insert your Token into Code => TOKEN
2. Add Bot to a Large Guild
   (I tested it on 3 Guilds with 60k, 50k and 50 Members. Bug only appears on Large Servers)
3. Give it proper Permissions to view Bans and Invites
4. Insert Guild ID of Large Guild => GUILD_ID_OF_LARGE_GUILD
5. Start Bot
6. See Bans and Invites appear in the console
7. Stop Bot
8. Remove Commenting on lines 45 and 49
9. Start Bot
10. Observe no output in the console
11. open ./Req.log
12. See Requests were sent
13. open ./Res.log
14. See Responses were received

Listening to <client>.rest.on('response', () => {}) breaks <guild>.bans.fetch() and <guild>.invites.fetch()

Code sample

const Discord = require('discord.js');

const client = new Discord.Client({
  shards: 'auto',
  partials: [],
  intents: new Discord.IntentsBitField(64),
});

client.login('TOKEN').then(() => {
  console.log('Logged in!');
});

client.on('ready', async () => {
  console.log(await client.guilds.cache.get('GUILD_ID_OF_LARGE_GUILD').bans.fetch());
  console.log(await client.guilds.cache.get('GUILD_ID_OF_LARGE_GUILD').invites.fetch());
});

const fs = require('fs');
const { Console } = require('console');

const reqOutput = fs.createWriteStream('./Req.log');
const reqErrorOutput = fs.createWriteStream('./reqErr.log');

const reqLogger = new Console({
  stdout: reqOutput,
  stderr: reqErrorOutput,
  ignoreErrors: true,
  colorMode: false,
});

const resOutput = fs.createWriteStream('./Res.log');
const resErrorOutput = fs.createWriteStream('./resErr.log');

const resLogger = new Console({
  stdout: resOutput,
  stderr: resErrorOutput,
  ignoreErrors: true,
  colorMode: false,
});

client.rest.on('request', (request) => {
  reqLogger.log(request);
});

/*
client.rest.on('response', (request, response) => {
  resLogger.log(response);
});
*/

Package version

[email protected]

Node.js version

v17.5.0

Operating system

Windows 11 Enterprise (64 Bit) & Ubuntu 20.04 (64 Bit)

Priority this issue should have

Medium (should be fixed soon)

Which partials do you have configured?

No Partials

Which gateway intents are you subscribing to?

Guilds, GuildBans, GuildInvites

I have tested this issue on a development release

4ba0f56

[almeidx]

Can replicate, even with d1ec8c37ffb7fe3b63eaa8c382f22ca1fb348c9b

[didinele]

Was going to say this is a duplicate, but I actually cannot find an open issue on this despite having discussed it with a maintainer before.

What was going on before is that, for large payloads, node-fetch's Response#clone() that was being used would actually end up causing the surpassing of the default highWaterMark, which causes underlying calls to methods such as json that consume the response body to never resolve, and therefore your API calls to never resolve. You can read about that here.

Now, with the introduction of undici, the clone call was instead changed to { ...res }, and I'm not particularly familiar with the behavior and intricacies of doing that, but I assume it's a similar underlying issue.

[KhafraDev]

Spreading into an empty object does a shallow clone. However, undici's Dispatcher.ResponseData (res) has a private symbol property that exposes the underlying stream, which is probably causing the same issue. Both node-fetch and undici.request rest implementation make use of node's streams under the hood, so it's safe to assume they'd both have the same bugs.