[BUG] [Errno 8] nodename nor servname provided, or not known

[101Dude]

Describe the bug.

I am getting random job errors for my jobs. My jobs run every 15 minutes and I get these errors only a few times per week. When they do happen it's most of the jobs if not all of them that are reporting the same error

[Errno 8] nodename nor servname provided, or not known

I'd be happy to help diagnose this and get it resolved on my side

Steps to reproduce. Running my webchanges jobs.yaml using crontab.

Expected behavior. I expect rules to run successfully every 15 minutes without generating an error occasionally while acknowledging that this is really a very small issue (which is why I have waited a year to file a bug report 😀)

Screen scrape/screenshots. output of webchanges -v attached

Version info.

09:28:25.897 iMac191 ~ 
➜ webchanges --detailed-versions
Software:
• webchanges: 3.25.0
• CPython: 3.12.0 ('main', 'Nov 21 2023 17:38:04') Clang 15.0.0 (clang-1500.0.40.1)
• SQLite: 3.43.2

System:
• Platform: macOS-14.6.1-x86_64-i386-64bit, x86_64
• Processor: i386
• CPUs (logical): 16
• Free memory: 98.6G physical plus 0.0B swap.
• Free disk '/': 129.0G (93.1%)

Installed PyPi dependencies:
• beautifulsoup4: 4.12.3
• chump: 1.6.0
• cssbeautifier: 1.15.1
  - jsbeautifier: 1.15.1
  - six: 1.16.0
  - editorconfig: 0.12.4
• cssselect: 1.2.0
• deepdiff: 7.0.1
  - ordered-set: 4.1.0
  - pyyaml: 6.0.2
• h2: 4.1.0
  - hyperframe: 6.0.1
  - hpack: 4.0.0
• html2text: 2024.2.26
• html5lib: 1.1
  - six: 1.16.0
  - webencodings: 0.5.1
  - lxml: 5.3.0
  - lxml: 5.3.0
• httpx: 0.27.0
  - anyio: 4.4.0
  - certifi: 2024.7.4
  - idna: 3.7
  - sniffio: 1.3.1
• jq: 1.8.0
• jsbeautifier: 1.15.1
  - six: 1.16.0
  - editorconfig: 0.12.4
• keyring: 25.3.0
  - jaraco.classes: 3.4.0
  - jaraco.functools: 4.0.2
  - jaraco.context: 5.3.0
• lxml: 5.3.0
• markdown2: 2.5.0
• msgpack: 1.0.8
• pdftotext: 2.2.2
• Pillow: 10.4.0
  - markdown2: 2.5.0
  - packaging: 24.1
  - typing-extensions: 4.12.2
• platformdirs: 4.2.2
• playwright: 1.46.0
  - greenlet: 3.0.3
  - pyee: 11.1.0
• psutil: 6.0.0
• pypdf: 4.3.1
  - typing_extensions: 4.12.2
  - wheel: 0.44.0
• pytesseract: 0.3.13
• pyyaml: 6.0.2
• redis: 5.0.8
  - typing-extensions: 4.12.2
  - requests: 2.32.3
• requests: 2.32.3
  - charset-normalizer: 3.3.2
  - idna: 3.7
  - urllib3: 2.2.2
  - certifi: 2024.7.4
  - PySocks: 1.7.1
• vobject: 0.9.7
  - python-dateutil: 2.9.0.post0

Playwright browser:
• Name: chromium
• Version: 128.0.6613.85
• Free memory with browser loaded: 98.4G physical plus 0.0B swap

Additional context. Howdy. I have been getting this error for all of my jobs when they run but randomly. My jobs are set to run every 15 minutes and this error only happens once in a while. I did a bit of googling and couldn't find a solution that I can implement. This has been happening many versions ago (possibly over a year now). I reported it to you awhile back. This page talks about how to fix it but I don't know if it's relevant to your code.

I am also running a VPN app with split tunneling. But this is on constantly and the jobs run without error while I am connected or disconnected - except as mentioned they occasionally return an error.

webchanges-v_out.txt

[mborsetti]

Hi @101Dude,

Thanks for the report! Learned a bit more about macOS errors, and it's cryptically reporting that a slow or unreachable DNS server isn't resolving the name to its IP address (servname [...] not known).

What DNS servers do you use?

If you're only using the public internet, you may want to try changing them to e.g. Cloudflare and/or Google servers, following Apple's instructions or one of the many tutorials. You can mix these servers for additional resiliency, e.g. use 8.8.8.8, 1.1.1.1, 2001:4860:4860::8888 and 2606:4700:4700::1111.

Please let me know in due time if that fixed the issue so this can be added to the documentation.

[101Dude]

I am using cloudflare DNS over HTTPS (DOH) on my router for WiFi and wired and my devices are set to pull the dns settings from the router at 192.168.10.1

1.1.1.1 and 1.0.0.1

[mborsetti]

That's good!

Only thing that comes to mind is to reduce concurrency by using --max-workers and specifying it to 12 or 8 threads; reducing concurrent calls to the resolution machinery hopefully will eliminate missed/delayed resolutions.

Another avenue is to look for some sort of adjustment in macOS for the timeout it uses to give up waiting and triggering [Errno 8] instead, and adjust its value (not familiar with the OS).

[101Dude]

I am now running a caching DNS server on a MacMini ( PiHole running in Docker using Cloudflare DOH ). I figured this should make all of the webchanges requests quick enough, since they are cached, that [Errno 8] won't recur.

I will let you know in a week if I have the same issue or not.

I've been wanting to do this for a few years and lacked motivation. Thanks for the inspiration!

[mborsetti]

Hello @101Dude,

Any updates on this?

Thanks.

[101Dude]

Hi, the update... I learned that my router was too slow to process the DOH requests for all of the devices in my house. I entered a manual DNS on the machine that I'm running webchanges on and that seems to have solved the issue. No errors since doing that.

I didn't test your recommendations for making changes to the webchanges configuration file.

Should I close this "issue"

Thank you for getting back to me!

Ps I ended up destroying the pihole because it didn't help the issue.

[mborsetti]

Great, thanks for the information. I will incorporate it into the docs.