`/core`: add an AsyncIterator-returning method for fetching members

[kyranet]

trafficstars

Which package is the feature request for?

core

Feature

The method would return an async iterator, which we would be able to use to consume data and process them as we receive them, rather than collecting all values into a collection and process them all at once.

The async iterator comes with a lot of benefits, one is that it allows us to process batches of up to 1000 members, distributing the load more evenly, e.g. creating classes from the members (e.g. discord.js), caching them somewhere (e.g. redis) or checking data from them (e.g. dehoisters, name search, etc).

for await (const member of client.requestGuildMembers({ guild_id })) {
	// ...
}

Ideal solution or implementation

Extend the work done in https://github.com/discordjs/discord.js/pull/8941, this time by implementing a method that returns an async iterator (we can have a private — or public — method that iterates over the raw batches, so we can minimize code duplication in the collect-all and the iterable method while maximizing performance.

The current method, requestGuildMembers, may be renamed.

Alternative solutions or implementations

N/a.

Other context

https://github.com/discordjs/discord.js/pull/8941#pullrequestreview-1229770370

[RealAlphabet]

If it is added, it would be interesting to do it for anything that has paging such as the ban list and audit logs. I think this should be part of a utility but can you imagine something like this?

Client.prototype.requestGuildMembers = async function* (options = {}) {
    let guild   = await Client.guilds.fetch(options.guildId);
    let after   = options.after     || "0";
    let limit   = options.chunkBy   || 1000;

    while (true) {
        let members = await guild.members.list({ limit, after });

        // Gives the relay to the iterator of the Collection to improve iteration performance.
        yield* members.values();

        if (members.size < limit)
            return;

        after = members.lastKey();
    }
};

I would love to create a new PR to implement this feature but I think it's better to wait for feedback on how the maintainers and community would like to see it implemented.

Personally, I think it would make sense to implement this in GuildMemberManager rather than Client. Or put it in a utility package rather than in /core.