10.0.0 Getting 429 Too Many Requests with Proxy

[SohelKabir]

Steps to reproduce

Use proxy agent with custom fetch implementation and will get 429 for multiple request.

Here's the code I'm using.

import { Innertube, UniversalCache, SearchFilters } from 'youtubei.js';

import { parseAboutData } from './utils';
import { HttpException } from '@nestjs/common';
import { HttpsProxyAgent } from 'https-proxy-agent';

import axios from 'axios';


const proxyUrl = 'http://username:[email protected]:12321';

export const initializeInnertube = async () => {
  const proxyAgent = new HttpsProxyAgent(proxyUrl);
  return await Innertube.create({
    cache: new UniversalCache(true),
    generate_session_locally: true,
    fetch: async (input, init) => {
      const url =
        typeof input === 'string'
          ? input
          : input instanceof URL
            ? input.toString()
            : input.url;
      const modifiedInit = {
        ...init,
        method: init?.method || (init?.body ? 'POST' : 'GET'),
         agent: proxyAgent,
      };
      if (modifiedInit.method === 'POST')
        modifiedInit.headers = {
          ...modifiedInit.headers,
          'Content-Type': 'application/json',
        };

      const maxRetries = 5;
      let retryCount = 0;
      const fetchWithRetry = async () => {
        try {
          const response = await fetch(url, modifiedInit);
          if (!response.ok) {
            if (response.status === 429 && retryCount < maxRetries) {
              retryCount++;
              const delayTime = Math.pow(2, retryCount) * 1000; // Exponential backoff
              console.warn(
                `Too Many Requests. Retrying in ${delayTime / 1000} seconds...`,
              );
              await delay(delayTime);
              return fetchWithRetry();
            }
            throw new Error(`Failed to fetch ${url}: ${response.statusText}`);
          }
          return response;
        } catch (error) {
          if (retryCount < maxRetries) {
            retryCount++;
            const delayTime = Math.pow(2, retryCount) * 1000; // Exponential backoff
            console.warn(
              `Fetch error for ${url}. Retrying in ${delayTime / 1000} seconds...`,
            );
            await delay(delayTime);
            return fetchWithRetry();
          }
          throw new Error(`Fetch error for ${url}: ${error.message}`);
        }
      };

      return fetchWithRetry();
    },
  });
};

Failure Logs

429 Too many requests.

Expected behavior

200

Current behavior

429 Too many requests.

Version

Default

Anything else?

No response

Checklist

• [X] I am running the latest version.
• [X] I checked the documentation and found no answer.
• [X] I have searched the existing issues and made sure this is not a duplicate.
• [X] I have provided sufficient information.