Crawling context sendRequest cookies can't be easily overriden

[MatousMarik]

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/core

Issue description

As shown in the API documentation but not explicitly stated anywhere, sendRequest takes an overrideOptions object that overrides options so that whatever you put into overrideOptions.headers.cookie or overrideOptions.cookieJar doesn't really matter if you won't also override overrideOptions.cookieJar.setCookie (or whatever internal function) to modify session cookies.

I think this should be explained somewhere here: https://crawlee.dev/docs/guides/got-scraping#cookiejar

My code: [email protected]:MatousMarik/send-request-bug.git Platform run: https://console.apify.com/actors/tIFxT28Ja3LFuVjkA/source

Code sample

// changes in cheerio ts template
import { CookieJar } from 'tough-cookie';
const crawler = new CheerioCrawler({
    proxyConfiguration,
    maxRequestsPerCrawl,
    requestHandler: async ({ request: { url }, sendRequest, session }) => {
        session?.setCookie('ahoj=svete', url);
        const cookieJar = new CookieJar();
        await cookieJar.setCookie(session!.getCookieString(url), url);
        await cookieJar.setCookie('hello=world', url);
        const response = await sendRequest({
            headers: {
                cookie: 'hi=world',
                Cookie: 'hello=world',
            },
            cookieJar,
        });
        console.log(response.request.options.headers);
        console.log('JAR:', await cookieJar.getCookieString(url));
    },
});

Package version

3.12.0

Node.js version

[email protected], [email protected]

Operating system

Sequoia 15.1 (24B83)

Apify platform

• [X] Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

[email protected]:MatousMarik/send-request-bug.git https://console.apify.com/actors/tIFxT28Ja3LFuVjkA/source