Issue creating a profile for an Intranet

[elvisvr]

Hi There, I getting an error trying to create a profile of a intranet website that uses NTLM authentication, I'm not sure whether that is possible at all, I'll appreciate any help? I'm using the following command: docker run -p 9222:9222 -p 9223:9223 -v profiles:/profiles -it webrecorder/browsertrix-crawler create-login-profile --interactive --url "http://tribunet/" --filename /profiles/profile_2.tar.gz and getting the following error: /app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:115 ? new Error(${response.errorText} at ${url}) ^ Error: net::ERR_NAME_NOT_RESOLVED at http://tribunet/ at navigate (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:115:23) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async FrameManager.navigateFrame (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:90:21) at async Frame.goto (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:416:16) at async Page.goto (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:789:16) at async main (/app/create-login-profile.js:127:3)

Thank you

[ikreymer]

It seems that the Docker container is unable to access the local DNS server. There are some solutions to this listed here using dnsmasq that should help: https://stackoverflow.com/questions/39400886/docker-cannot-resolve-dns-on-private-network

What OS are you running on? If you're on linux, you may also be able to use host networking by adding docker run --network host ... when creating the profile (and you'll of course also need this when running the crawl).

I'd be interested to know if either of these solutions work!

[elvisvr]

Hi, thank you for your prompt reply. I'm running on Windows 10 PRO, I'll follow your link suggestions and hopefully it will work.

[despens]

I would recommend using docker-compose and the extra_hosts option:

It could look something like this:

version: '3.5'
services:
    crawler:
        image: webrecorder/browsertrix-crawler:latest
        build:
          context: ./
        volumes:
          - ./crawls:/crawls
        cap_add:
          - NET_ADMIN
          - SYS_ADMIN
        shm_size: 1gb
        extra_hosts:
            "intranet.local": 192.168.1.2