Add custom headers

[Mecanik]

Awesome library!

Could you add support to add custom headers? It is quite a requirement to add custom user agent / referrer because many websites will block you from scraping too often.

[vantoozz]

Hi @Mecanik!

It is possible now to set custom headers on the HTTP client level; you can do something like this:

<?php declare(strict_types=1);

use GuzzleHttp\Client;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\RequestOptions;
use Psr\Http\Message\RequestFactoryInterface;
use Psr\Http\Message\RequestInterface;
use Vantoozz\ProxyScraper\HttpClient\PsrHttpClient;

use function Vantoozz\ProxyScraper\proxyScraper;

require_once __DIR__ . '/vendor/autoload.php';

$httpClient = new PsrHttpClient(
    new Client([
        RequestOptions::CONNECT_TIMEOUT => 10,
        RequestOptions::TIMEOUT => 20,
    ]),
    new class implements RequestFactoryInterface {
        public function createRequest(
            string $method,
            $uri
        ): RequestInterface {
            return (new Request($method, $uri))
                ->withHeader('User-Agent', 'MyUserAgent')
                ->withHeader('Referer', 'https://example.com/');
        }
    }
);

$scraper = proxyScraper($httpClient);

This will add custom headers for all the requests