stdout differences between locales

[ctshanghai]

We came across an issue where output is unique with LANG=en_US which is causing issues with automation. While we can work around it by forcing LANG=en_US it's still vexing.

# python --version
Python 3.8.0

# locale -a
C
C.UTF-8
en_US
en_US.iso88591
en_US.utf8
POSIX

Here is what we expect:

# export LANG=en_US
# aiodnsbrute --no-verify --gethostbyname -f - -o json example.org
[{"domain": "www.example.org", "ip": ["93.184.216.34"]}]

When LANG is not one of the en* locales, things I expect to be on stderr come out on stdout.

# export LANG=C.UTF-8
# aiodnsbrute --no-verify --gethostbyname -f - -o json example.org
[*] Brute forcing example.org with a maximum of 512 concurrent tasks...
[!] Skipping domain verification. YOLO!
[*] Using recursive DNS with the following servers: ['8.8.8.8', '8.8.4.4', '114.114.114.114']
[*] No wildcard response was detected for this domain.
[*] Using pycares `gethostbyname` function to perform lookups, CNAME data will be appended to results (** denotes CNAME, show actual name with -vv)
[*] Wordlist loaded, proceeding with 1000 DNS requests
[+] www.example.org                             ['93.184.216.34']                                                                                                                                                                                 
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:06<00:00, 163.37rec/s]
[*] Completed, 1 subdomains found
# export LANG=en_US.utf8
# aiodnsbrute --no-verify --gethostbyname -f - -o json example.org
[{"domain": "www.example.org", "ip": ["93.184.216.34"]}]

Is this an issue in aiodnsbrute, or an issue in an underlying library used, or a problem between monitor & chair?

[suhongrui]

I also have the same problem.

[blark]

I will take a look a this, thanks for the report - sorry for the issue!

[blark]

@suhongrui @ctshanghai can you please try out the dev branch? I think I fixed it.