macOS Net::ReadTimeout

[ethicalhack3r]

Getting a Net::ReadTimeout error on macOS running arachni-1.5-0.5.11

Full error log: https://gist.github.com/ethicalhack3r/3bf395dfdbb3fae9cd13a14c977fcd7c

/etc/hosts file has this entry:

127.0.0.1 mywordpress.com

I can access mywordpress.com from a browser no problem. I tried adding a proxy to the profile to see if any requests were going through and no requests were seen in the proxy.

Ruby version:

$ ruby -v
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin16]

(also tried with Ruby 2.2.0 and get same error)

macOS version: macOS Sierra version 10.12.3 (16D32)

Do you have any suggestions?

[Zapotek]

Can you give the nightlies a try please?

[ethicalhack3r]

Same error using the nightlies (Arachni v2.0dev - WebUI v1.0dev)

[ethicalhack3r]

Same error when trying to scan a domain on the Internet (rather than localhost). That should rule out the target being on localhost as the culprit.

[Zapotek]

This doesn't have anything to do with the target, Selenium can't connect to PhantomJS on initialization, which is very strange because there's no reason for it.

Let me try it on macOS Sierra and get back to you.

[Zapotek]

Couldn't reproduce, do you happen to have anything else tricky in your hosts file or network interface setup? Also, could you please try the following and upload the output?

./bin/arachni_console
> debug_on 5
> Browser.new
> exit

And:

./bin/arachni_console
> debug_on 5
> BrowserCluster.new
> exit

[ethicalhack3r]

/etc/hosts has these other entries which I think are default:

127.0.0.1	localhost
255.255.255.255	broadcasthost
::1             localhost 
fe80::1%lo0	localhost

Browser.new output: https://gist.github.com/ethicalhack3r/8d91e483703b21a11353b9de8729e5ad

BrowserCluster.new output: https://gist.github.com/ethicalhack3r/a324bad11480f0d331b0100122adbf9a

(both give the same error)

[Zapotek]

I'm at a bit of a loss, can you try increasing the timeout with --browser-cluster-job-timeout=60 please?

[ethicalhack3r]

I get Unrecognized switch error when running $ ./bin/arachni_console --browser-cluster-job-timeout=60

Assuming the switch isn't for arachni_console?

[Zapotek]

No it's for ./bin/arachni, for the console you can run:

> Options.browser_cluster.job_timeout = 60
> debug_on 5
> Browser.new

[ethicalhack3r]

Seems to work! https://gist.github.com/ethicalhack3r/3c4c835ed7272d8a0dec1ff62ff6b62c

[Zapotek]

Great, I'll update the defaults with a higher timeout and push nightlies. Thanks for the feedback.

[Zapotek]

Btw, could you do me a favor? Can you try decreasing the timeout by 10 and see when it stops working for you?

[ethicalhack3r]

Sorry! I spoke a few seconds too soon. :( Same error if I actually wait longer than 60 seconds.

Tried 120 seconds with the same Net::ReadTimeout error.

[Zapotek]

Hm, I'll prepare a script for you to try and pinpoint down the issue.

[Zapotek]

In terminal 1 execute:

./bin/arachni_shell -c "phantomjs --webdriver=9999"

Wait until the following appears:

[INFO  - 2017-02-17T14:59:16.768Z] GhostDriver - Main - running on port 9999

If it takes too long please let me know.

Create file client.rb:

uri = ::URI.parse( 'http://localhost:9999/' )
p Net::HTTP.get( uri )

In terminal 2 execute:

./bin/arachni_script client.rb

Finally please provide the output of terminal 2.

[ethicalhack3r]

Sorry for the delayed response.

Terminal 1 output: https://gist.github.com/ethicalhack3r/04475775e09783e1625c193e54c4649f

Terminal 2 output: https://gist.github.com/ethicalhack3r/70bf2e2b15a68eaf22dae92e501f2355

[Zapotek]

I was hoping for a timeout error again, that would have made some sense.

Can you try the following as the client script please?

client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = Options.browser_cluster.job_timeout

Selenium::WebDriver.for( :remote,
    url:         'http://localhost:9999/',
    http_client: client
)

This more closely resembles what the system does. If that doesn't help either I've got one last idea.