apify
sendRequest not using current session
Which package is this bug report for? If unsure which one to select, leave blank
None
Issue description
after release v3.13.7 sendRequest not using the session from the navigation anymore.
the cookie options inside https://github.com/apify/crawlee/blob/master/packages/basic-crawler/src/internals/send-request.ts#L30-L36 is undefined after this commit.
is this expected?
Code sample
const crawler = new CheerioCrawler({
/** ... **/
requestHandler: ({ sendRequest, addRequests }: CheerioCrawlingContext) => {
const { body: captchaImage } = await sendRequest({}); // get captcha image
// solve captcha here
await addRequests([
/** another request after captcha already solved **/
]);
},
});
await crawler.run(['https://example.com']);
Package version
v3.13.7
Node.js version
22
Operating system
debian
Apify platform
- [ ] Tick me if you encountered this issue on the Apify platform
I have tested this on the next release
No response
Other context
No response
This is a know issue (#2917), basic crawler has no notion of a session or proxy.
Hmm, maybe I misunderstood what you are reporting. Which is quite normal, given there is no reproduction. Please provide one so its clear what you are doing.
Sorry if it is not clear enough, and thank you for the fast response.
Let's say i want to solve captcha in one of my crawler.
const crawler = new CheerioCrawler({
/** ... **/
requestHandler: ({ sendRequest, addRequests }: CheerioCrawlingContext) => {
const { body: captchaImage } = await sendRequest({}); // get captcha image
// solve captcha here
await addRequests([
/** another request after captcha already solved **/
]);
},
});
await crawler.run(['https://example.com']);
With crawlee before v3.13.7, i can handle my use case from the code above. For now i am using my own GotScrapingHttpClient, but without the commit from a2985f259f068fbe00aed931a812b8a8755282cb
