projectdiscovery
fix stdin parsing
This pull request breaks -vhost-input for me, see https://github.com/projectdiscovery/httpx/issues/2020
If it helps my understanding of the feature is this:
Sometimes you have hosts that you need to specify by IP, to do vhost discovery. (Send different Hosts: header for the same IP)
You might also want to process multiple hosts / vhosts in one session without starting up multiple HTTPX.
So you specify each vhost/IP pair on stdin like this: cat inputfile.txt | httpx -vhost-input -title
You would of course have a input file with multiple vhosts you want to try on an IP (and multiple hosts you want to test) like this:
inputfile.txt
example.org,https://93.184.215.14
example2.org,https://93.184.215.14
www.google.com,https://74.125.68.147
This should give output like this:
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_|
/_/
projectdiscovery.io
[INF] Current httpx version v1.6.9 (latest)
[WRN] UI Dashboard is disabled, Use -dashboard option to enable
https://93.184.215.14 [Example Domain]
https://74.125.68.147 [Google]
http://93.184.215.14 [404 - Not Found]
@noaho I've made some updates. Please give it another try and share your thoughts. Thanks!
Hi @dogancanbakir,
I tried the changes with both formats and it seems the new format isn't passing through the URL properly - the end web server seems to see only IP address, no Host: or sni.
While the old format seems to ignore the scheme, and always probe with https.
go install github.com/projectdiscovery/httpx/cmd/httpx@4dc7f90 # (this is the right commit, right? it's the last one you did)
# Make sure you re-resolve example.org if stealing my examples, the IP did change for me once
cat inputfile.txt
example.org,http://23.215.0.133
example.org,https://23.215.0.133
cat inputfile2.txt
http://example.org[23.215.0.133]
https://example.org[23.215.0.133]
cat inputfile.txt | httpx -vhost-input -json -no-fallback-scheme | jq
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_|
/_/
projectdiscovery.io
[INF] Current httpx version v1.6.9 (latest)
[WRN] UI Dashboard is disabled, Use -dashboard option to enable
{
"timestamp": "2025-01-21T03:49:49.243668565Z",
"port": "443",
"url": "https://23.215.0.133",
"input": "example.org,https://23.215.0.133",
"title": "Example Domain",
"scheme": "https",
"content_type": "text/html",
"method": "GET",
"host": "23.215.0.133",
"path": "/",
"time": "32.104988ms",
"a": [
"23.215.0.133"
],
"tech": [
"HTTP/3"
],
"words": 298,
"lines": 46,
"status_code": 200,
"content_length": 1256,
"failed": false,
"knowledgebase": {
"PageType": "nonerror",
"pHash": 0
}
}
{
"timestamp": "2025-01-21T03:49:49.24683302Z",
"port": "443",
"url": "https://23.215.0.133",
"input": "example.org,http://23.215.0.133",
"title": "Example Domain",
"scheme": "https",
"content_type": "text/html",
"method": "GET",
"host": "23.215.0.133",
"path": "/",
"time": "34.903798ms",
"a": [
"23.215.0.133"
],
"tech": [
"HTTP/3"
],
"words": 298,
"lines": 46,
"status_code": 200,
"content_length": 1256,
"failed": false,
"knowledgebase": {
"PageType": "nonerror",
"pHash": 0
}
}
Note the scheme is https, twice. It didn't try on http.
cat inputfile2.txt | httpx -vhost-input -json -no-fallback-scheme | jq
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_|
/_/
projectdiscovery.io
[INF] Current httpx version v1.6.9 (latest)
[WRN] UI Dashboard is disabled, Use -dashboard option to enable
{
"timestamp": "2025-01-21T03:51:23.06037323Z",
"port": "80",
"url": "http://23.215.0.133",
"input": "http://example.org[23.215.0.133]",
"title": "Invalid URL",
"scheme": "http",
"webserver": "AkamaiGHost",
"content_type": "text/html",
"method": "GET",
"host": "23.215.0.133",
"path": "/",
"time": "7.496144ms",
"a": [
"23.215.0.133"
],
"words": 8,
"lines": 8,
"status_code": 400,
"content_length": 310,
"failed": false,
"knowledgebase": {
"PageType": "error",
"pHash": 0
}
}
{
"timestamp": "2025-01-21T03:51:23.074694035Z",
"port": "443",
"url": "https://23.215.0.133",
"input": "https://example.org[23.215.0.133]",
"title": "Invalid URL",
"scheme": "https",
"webserver": "AkamaiGHost",
"content_type": "text/html",
"method": "GET",
"host": "23.215.0.133",
"path": "/",
"time": "21.762277ms",
"a": [
"23.215.0.133"
],
"words": 8,
"lines": 8,
"status_code": 400,
"content_length": 310,
"failed": false,
"knowledgebase": {
"PageType": "error",
"pHash": 0
}
}
Note it didn't pass the correct Host: or SNI
Cheers!
the other problem I have with this functionality is when you use the screenshot function, it saves the screenshots as the IP address only, so when you have multiple vhosts/IP pairs it just overwrites and corrupts them after the first one..
@dogancanbakir I tried the latest change (set customip) and it might be working but it breaks -proxy.. It seems to try to connect to the passed vhost on the proxy port as the proxy..
Was trying to use burp suite to verify that your changes covered all cases and then came across that issue.
inputfile2.txt
http://www.example.org[23.205.104.21]
Success:
cat inputfile2.txt | httpx -json -no-fallback-scheme -debug -vhost-input
[INF] Current httpx version v1.6.9 (outdated)
[WRN] UI Dashboard is disabled, Use -dashboard option to enable
[INF] Dumped HTTP request for http://www.example.org
GET / HTTP/1.1
Host: www.example.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1.1 Safari/605.1.15
Accept-Charset: utf-8
Accept-Encoding: gzip
<snip>
{"timestamp":"2025-02-18T08:10:39.204453175Z","port":"80","url":"http://www.example.org","input":"http://www.example.org[23.205.104.21]","title":"Example Domain","scheme":"http","content_type":"text/html","method":"GET","host":"23.205.104.21","path":"/","time":"10.905078ms","a":["23.218.217.177","23.218.217.154"],"aaaa":["2600:1408:ec00:36::1736:7f24","2600:1408:ec00:36::1736:7f2e"],"cname":["www.example.org-v2.edgesuite.net","a1519.dscr.akamai.net"],"words":298,"lines":46,"status_code":200,"content_length":1256,"failed":false,"knowledgebase":{"PageType":"nonerror","pHash":0},"resolvers":["1.1.1.1:53","1.0.0.1:53"]}
But with -proxy:
cat inputfile2.txt | httpx -json -no-fallback-scheme -debug -vhost-input -proxy http://127.0.0.1:8080
[INF] Current httpx version v1.6.9 (outdated)
[WRN] UI Dashboard is disabled, Use -dashboard option to enable
[INF] Dumped HTTP request for http://www.example.org
GET / HTTP/1.1
Host: www.example.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36
Accept-Charset: utf-8
Accept-Encoding: gzip
[DBG] Failed 'http://www.example.org': GET http://www.example.org giving up after 1 attempts: Get "http://www.example.org": read tcp <MY_IP>:39920->23.205.104.21:8080: read: connection reset by peer
Note that it connected to port 8080, which is the port of my proxy, not http://www.example.org
Also it seems to be always resolving DNS, even when we provide the IP we want to connect on. I'm not sure if that's a bug or not, but I was trying to verify that it will connect to the provided IP with burp (and not just the one in DNS)
This pull request has been automatically marked as stale due to inactivity. It will be closed in 7 days if no further activity occurs. Please update if you wish to keep it open.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}