emersion
Set read/write timeout per read/write call on the underlying connection, not the command
go-imap client sets deadlines for writing the whole command and reading a whole response. Sometimes response can be large and I am on a slow connection but I still receive a few bytes every few seconds, but then connection drops 30 seconds later because the full response did not arrive within 30 seconds.
If you find a way to set the timeout on a read() and write() syscalls or as close as possible to them, that would be better. This is how we do it in Delta Chat. We establish a TCP connection with a dialing timeout, then set up read and write timeout on the stream (all timeouts are generous 60 seconds) and never worry about them except when using IDLE. When we use IDLE, we remove read timeout and put the timeout around the IDLE command, then immediately put it back on the underlying stream.
This way we never have to worry about having some code that might forget to put the timeout while writing or reading into the channel. All timeouts look like a stream failure similar to "connection reset by peer". We never pattern-match on errors, so this terminates the connection and we open a new one (with some backoff to avoid getting into reconnection loop in case of repeated failures).
You can probably do something similar, instead of using DialTLS, establish a connection, then wrap it into TLS, set read/write timeouts using Conn.SetDeadline() and after that call New(). Inside a wrapper, reset the deadline of the underlying connection on every read/write call.
From a quick search, this is a known problem in Go HTTP library that you cannot set the timeouts and have to reset the deadline each time but if you are on a high level then you can only do it per HTTP request/response: https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/ This is a similar problem, you only reset deadlines per IMAP command/response, so a large FETCH may timeout even if read() syscalls succeed all the time.
This is intentional. Individual responses are single-line and should each be written in a single step. Having per-read/write timeouts would allow the server/client to stretch the timeout by sending one byte at a time. go-imap has more permissive timeouts for literals and such.
What about FETCH response? I see timeout is reset here:
https://github.com/emersion/go-imap/blob/5d38391f8fbe28dd9a83c65281792446fd83af2d/imapclient/fetch.go#L815-L822
But if I request a message body and the message is just large, the whole message should be fetched in 30 seconds?
Having per-read/write timeouts would allow the server/client to stretch the timeout by sending one byte at a time. go-imap has more permissive timeouts for literals and such.
If the server is malicious, it can do much worse, e.g. serve me incorrect responses or large number of unsolicited responses without ever responding to the command. And still, if I need to interact with such malicious servers, I can additionally set timeouts inside the app for how long I want to wait for getting the result before deciding it's not worth trying.
Slow internet, on the other hand, is a much more common problem than malicious servers running reverse "slowloris" attacks against the clients. See also https://brr.fyi/posts/engineering-for-slow-internet
A single FETCH response item can take up to 30s. A FETCH response literal can take 5min. These timeouts sound reasonable to me.