foxr
foxr copied to clipboard
Bug: concurrent crawling
I've been playing around with foxr to find out if I can use it as a substitute of puppeteer for firefox-based browsers. I tried this code :
const foxr = require("foxr").default;
async function main() {
const browser = await foxr.launch({
headless: false,
args: [
"-marionette",
"-no-remote",
"-profile",
"/tmp/firefox_9a139bbd688bad29dce5b4fd"
],
executablePath: "/usr/bin/firefox"
});
const urls = ["http://google.com", "http://youtube.com", "http://facebook.com"]
//await lib.crawl(browser, urls);
const page1 = await browser.newPage();
const page2 = await browser.newPage();
const page3 = await browser.newPage();
await Promise.all([
page1.goto(urls[0]),
page2.goto(urls[1]),
page3.goto(urls[2])
]);
}
main();
Expected behavior : google, youtube and facebook open in three different tabs (code works with puppeteer) Observer behavior: I get two empty tabs, and one tab with facebook. Context : firefox 67.0.2 on ubuntu, foxr 0.9.0
I also noticed that firefox closes on its own after a few seconds of inactivity. Is that an effect of -marionette
?
code works with puppeteer
unfortunately that's not how Marionette works – there is no way to send command to particular page by internal ID or something, it affects only a "current" page.
you might want to give puppeteer-firefox a try (keep in mind that it downloads a custom-built Firefox
)
firefox closes on its own after a few seconds of inactivity
that shouldn't really happen, at least not in a few seconds. any simple way to reproduce it?
unfortunately that's not how Marionette works – there is no way to send command to particular page by internal ID or something, it affects only a "current" page.
Too bad, I will have to use puppeteer-firefox. I wanted to avoid it because like you said, it's a custom-built Firefox
that shouldn't really happen, at least not in a few seconds. any simple way to reproduce it?
const foxr = require("foxr").default;
async function main() {
const browser = await foxr.launch({
headless: false,
args: [
"-marionette",
"-no-remote",
"-profile",
"/tmp/firefox_9a139bbd688bad29dce5b4fd"
],
executablePath: "/usr/bin/firefox"
});
}
main();
With this code, Firefox starts, then closes after about 10 seconds on my PC.
With this code, Firefox starts, then closes after about 10 seconds on my PC.
I've figured it out – actually it's "by design". Whenever Node.js has nothing more to do it quits, and by default Foxr is trying to cleanup and stop Firefox processes it has launched.