broken-link-checker icon indicating copy to clipboard operation
broken-link-checker copied to clipboard

Requests would be hanging since there is no configuration for response timeout

Open jetlv opened this issue 7 years ago • 4 comments

Hi Steven,

I am using your awesome module to develop some api services. But recently my api hangs while facing some target web links which can not provide response in time. Then I searched your provide api documentation, no timeout value could be configured.

I was just debugging your source code, then noticed that you were using node-bhttp as dependency. I checked the request options you were using -

{
		discardResponse: true,
		headers: { "user-agent":options.userAgent },
		method: retry!==405 ? options.requestMethod : "get"
}

There is no timeout set. I've checked the bhttp doc - https://github.com/joepie91/node-bhttp

Seems there is an advanced options that can be used,

responseTimeout: The timeout, in milliseconds, after which the request should be considered to have failed if no response is received yet. Note that this measures from the start of the request to the start of the response, and is not a connection timeout. If a timeout occurs, a ResponseTimeoutError will be thrown asynchronously (see error documentation below).

Also there are some more references -

bhttp.ConnectionTimeoutError

The connection timed out.

The connection timeout is defined by the operating system, and cannot currently be overridden.

bhttp.ResponseTimeoutError

The response timed out.

The response timeout can be specified using the responseTimeout option, and it is measured from the start of the request to the start of the response. If no response is received within the responseTimeout, a ResponseTimeoutError will be thrown asynchronously, and the request will be aborted.

You should not set a responseTimeout for requests that involve large file uploads! Because a response can only be received after the request has completed, any file/stream upload that takes longer than the responseTimeout, will result in a ResponseTimeoutError.

Could you please help on this? I only expect that my api won't hang even if it's facing some strange links.

Thank you very much!

jetlv avatar Mar 20 '17 06:03 jetlv

The timeout is defined by the operating system and is usually 20,000ms. Setting Node's http timeout to anything longer than that will not override the OS setting. Setting to anything shorter might be insufficient.

stevenvachon avatar Mar 20 '17 14:03 stevenvachon

For some reason, one of the links here (https://jenkins.io/blog/2017/04/20/secure-jenkins-on-azure/) will take forever to be checked. I ran it as a background job and after 2 days it was still running. What can be the issue? Why there was no timeout? How can I see what is being checked right now?

blc https://jenkins.io/blog/2017/04/20/secure-jenkins-on-azure/

Looks like it is related to http://www.putty.org/. There is a link to this site at Jenkins blog post. And it hangs also:

blc http://www.putty.org/

vkotovv avatar Jul 26 '17 10:07 vkotovv

@stevenvachon any idea why is this happening?

vkotovv avatar Jul 31 '17 08:07 vkotovv

Hello! I am having the same problem it hangs at a certain url and never ends waiting for a resoponse from the other side, I tried excluding the url that is giving a hang but excluding that certaing URL will make my action specific for that URL and if another URL like this one comes the same thing will happen again. Is the response time-out bug been resolved. If it has then what is the syntax for me to pass the time-out value, I did not find anything related to time-out feature in --help utility

HarshalRathore avatar Sep 24 '21 13:09 HarshalRathore