How do I use proxies

[XxHiroTTVxX]

I get 400 bad request but I'm handling it correctly

[vakhov]

Hi there!

Thank you for your interest in the project! A 400 Bad Request error can happen for several reasons. Could you provide more details about the following:

• How are you using the proxies? (e.g., in which application/library and how you configure them)
• What headers or parameters are being sent in your request?
• Does the error occur consistently or only with specific proxies?

If you're confident everything is set up correctly, the issue might be related to the proxies themselves (e.g., they could be blocked, require authentication, or are outdated).

It would be helpful if you could share a code snippet or example that reproduces the error. This will make it easier to understand the problem and assist you further.

[XxHiroTTVxX]

so I have this

import fs from 'fs';

// Test script to diagnose proxy issues
async function testProxyRequest(proxyUrl: string, targetUrl: string) {
    try {
      console.log(`Testing proxy ${proxyUrl} to ${targetUrl}`);
      
      // Test with different header combinations
      const headers = {
        // Basic headers
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
        
        // Additional headers that might help
        'Accept': '*/*',
        'Accept-Language': 'en-US,en;q=0.9',
        'Connection': 'keep-alive'
      };
  
      // Try request with basic headers
      const basicResponse = await fetch(targetUrl, {
        headers: {
          'User-Agent': headers['User-Agent']
        },
        proxy: proxyUrl
      });
      console.log('Basic headers result:', {
        status: basicResponse.status,
        statusText: basicResponse.statusText
      });
  
      // Try with all headers
      const fullResponse = await fetch(targetUrl, {
        headers,
        proxy: proxyUrl
      });
      console.log('Full headers result:', {
        status: fullResponse.status,
        statusText: fullResponse.statusText
      });
  
      // Try without any headers
      const noHeaderResponse = await fetch(targetUrl, {
        proxy: proxyUrl
      });
      console.log('No headers result:', {
        status: noHeaderResponse.status,
        statusText: noHeaderResponse.statusText
      });
  
    } catch (error) {
      console.error('Error during test:', error);
    }
  }
  
  // Example usage:
  const testUrls = [
    'https://graphql.anilist.co',
    'https://anitaku.bz'
  ];
  
  async function runTests() {
    // Read proxies from proxies.json
    const proxiesData = fs.readFileSync('proxies.json', 'utf-8');
    const proxies = JSON.parse(proxiesData);

    // Test each proxy with each URL
    for (const proxy of proxies) {
      for (const url of testUrls) {
        console.log(`\n=== Testing ${url} ===`);
        await testProxyRequest(proxy.proxyUrl, url);
      }
    }
  }
  
  runTests();```


and when I run that I get this


=== Testing https://anitaku.bz ===
Testing proxy http://159.112.235.80:80 to https://anitaku.bz
Basic headers result: {
  status: 400,
  statusText: "Bad Request",
}
Full headers result: {
  status: 400,
  statusText: "Bad Request",
}
No headers result: {
  status: 400,
  statusText: "Bad Request",
}

=== Testing https://graphql.anilist.co ===
Testing proxy http://172.67.142.195:80 to https://graphql.anilist.co
Basic headers result: {
  status: 400,
  statusText: "Bad Request",
}
Full headers result: {
  status: 400,
  statusText: "Bad Request",
}
No headers result: {
  status: 400,
  statusText: "Bad Request",
}


not this is just a test code snipit 

[vakhov]

Hi again,

Thanks for providing the detailed test code and results! It seems like the issue is consistent across both target URLs and all proxy requests, regardless of the headers being used. Here are some points to investigate further:

1. Proxy Validation:
   Make sure the proxies in your proxies.json file are valid and functional. You can use an external tool or service to test the proxies independently. Sometimes, proxies can be outdated, blocked, or restricted by the target server.

2. Target URL Restrictions:
   Some websites (like anitaku.bz or graphql.anilist.co) may block traffic from certain proxies or enforce strict checks on incoming requests. It's worth checking if these websites have specific anti-bot protections or region restrictions that could cause the 400 Bad Request error.

3. Proxy Support in fetch:
   Are you using a library or wrapper that supports proxies with fetch? Native fetch in Node.js does not have built-in proxy support. You may need to use a library like [node-fetch-proxy](https://www.npmjs.com/package/node-fetch-proxy) or [axios](https://www.npmjs.com/package/axios) with proxy settings to ensure your requests are routed through the proxy correctly.

4. Request Format:
   Confirm that the target URLs are expecting a GET request with no additional body. If these endpoints require specific query parameters or authentication tokens, they may respond with 400 Bad Request for missing or incorrect data.

5. Debug the Proxies:
   Try using a simple curl command with the same proxies to verify if they work as expected:

   curl -x http://159.112.235.80:80 https://anitaku.bz
   

   If curl also fails, it's likely an issue with the proxy itself or how the target server interacts with it.

6. Headers and Connection Type:
   Sometimes, adding Host or Origin headers might help.

[XxHiroTTVxX]

Okay so I'll try some of that stuff out how can I make sure all the proxies are valid and for the fetch I'm using bun.sh

[XxHiroTTVxX]

I just tried curl

curl -x 173.245.49.196:80 https://anilist.co

curl: (56) Received HTTP code 400 from proxy after CONNECT

[XxHiroTTVxX]

for the one at the very bottom works in http.ts is there something I can do to get the latest proxies and just test those ? like last 10 days or last 5 days depending on how much work

[XxHiroTTVxX]

I also noticed that Anitaku gets 200 then 522 what is that about ```sh Testing 111.1.61.47:3128... [fetch] > HTTP/1.1 POST https://anitaku.bz/ [fetch] > content-type: application/json [fetch] > host: anitaku.bz [fetch] > origin: https://anitaku.bz [fetch] > Connection: keep-alive [fetch] > User-Agent: Bun/1.2.2 [fetch] > Accept: / [fetch] > Accept-Encoding: gzip, deflate, br [fetch] > Content-Length: 52

[fetch] < 200 Connection established

[fetch] < 522 [fetch] < Date: Mon, 10 Feb 2025 00:14:17 GMT [fetch] < Content-Type: text/html; charset=UTF-8 [fetch] < Content-Length: 7050 [fetch] < Connection: keep-alive [fetch] < Report-To: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=vPRB2mvwpo3RpGkaFXEaN6VlHAPDTFzDItIJgu3llh6bWvDqjAwwGw7xi8hbWfeE1tslCUkq4SFmlN5giUQkSNEiru2Fg%2FT4GVDyKa2aXn3qzRJTs%2BopnB7LsbwQ"}],"group":"cf-nel","max_age":604800} [fetch] < NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800} [fetch] < X-Frame-Options: SAMEORIGIN [fetch] < Referrer-Policy: same-origin [fetch] < Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 [fetch] < Expires: Thu, 01 Jan 1970 00:00:01 GMT [fetch] < Server: cloudflare [fetch] < CF-RAY: 90f7e4f9da32b9ac-SEA [fetch] < alt-svc: h3=":443"; ma=86400 [fetch] < server-timing: cfL4;desc="?proto=TCP&rtt=176299&min_rtt=175920&rtt_var=37482&sent=5&recv=7&lost=0&retrans=0&sent_bytes=3182&recv_bytes=912&delivery_rate=24687&cwnd=252&unsent_bytes=0&cid=32176e8a4d412a09&ts=39961&x=0"```

[XxHiroTTVxX]

I also noticed that Anitaku gets 200 then 522 what is that about

Testing 111.1.61.47:3128...
[fetch] > HTTP/1.1 POST https://anitaku.bz/
[fetch] > content-type: application/json
[fetch] > host: anitaku.bz
[fetch] > origin: https://anitaku.bz
[fetch] > Connection: keep-alive
[fetch] > User-Agent: Bun/1.2.2
[fetch] > Accept: */*
[fetch] > Accept-Encoding: gzip, deflate, br
[fetch] > Content-Length: 52

[fetch] < 200 Connection established

[fetch] < 522 
[fetch] < Date: Mon, 10 Feb 2025 00:14:17 GMT
[fetch] < Content-Type: text/html; charset=UTF-8
[fetch] < Content-Length: 7050
[fetch] < Connection: keep-alive
[fetch] < Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=vPRB2mvwpo3RpGkaFXEaN6VlHAPDTFzDItIJgu3llh6bWvDqjAwwGw7xi8hbWfeE1tslCUkq4SFmlN5giUQkSNEiru2Fg%2FT4GVDyKa2aXn3qzRJTs%2BopnB7LsbwQ"}],"group":"cf-nel","max_age":604800}
[fetch] < NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
[fetch] < X-Frame-Options: SAMEORIGIN
[fetch] < Referrer-Policy: same-origin
[fetch] < Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
[fetch] < Expires: Thu, 01 Jan 1970 00:00:01 GMT
[fetch] < Server: cloudflare
[fetch] < CF-RAY: 90f7e4f9da32b9ac-SEA
[fetch] < alt-svc: h3=":443"; ma=86400
[fetch] < server-timing: cfL4;desc="?proto=TCP&rtt=176299&min_rtt=175920&rtt_var=37482&sent=5&recv=7&lost=0&retrans=0&sent_bytes=3182&recv_bytes=912&delivery_rate=24687&cwnd=252&unsent_bytes=0&cid=32176e8a4d412a09&ts=39961&x=0"

[2noScript]

I have the same and know how to use proxy