Connection can't be closed when there is unread `DataFrame` in the stream

[bitoku]

When a stream gets a DataFrame and nothing reads data from the stream of the connection, it can't close the connection and leads to memory leak.

When I'm working on https://github.com/cri-o/cri-o memory leak issue, I found that in the situation above, spdy stream server causes memory leak. The leak happens when it tries to close the connection, and it waits wg.Wait() forever.

https://github.com/moby/spdystream/blob/57d1ca2bd6b9a33c1ac755d5f8033d0abdca3324/connection.go#L392-L394

The cause of the issue is that it can't finish frameHandler() because of the deadlock. There are some workers calling frameHandler(), and wg.Add() when each worker starts and wg.Done() when each worker is done.

https://github.com/moby/spdystream/blob/57d1ca2bd6b9a33c1ac755d5f8033d0abdca3324/connection.go#L317-L333

DataFrames queued in a PriorityFrameQueue are processed in frameHandler() and if it is a DataFrame, it is handled by dataFrameHandler() ( = handleDataFrame()). In the function, it blocks until either the stream is closed or something reads the stream dataChan (e.g. something calls Read() or ReadData()) Otherwise the function can't finish, which means frameHandler() also can't finish, and it never calls wg.Done().

https://github.com/moby/spdystream/blob/57d1ca2bd6b9a33c1ac755d5f8033d0abdca3324/connection.go#L598-L607

If nothing reads the data from the stream, the only way of blocking it is to close the stream. However streams in a connection are closed after wg.Wait().

https://github.com/moby/spdystream/blob/57d1ca2bd6b9a33c1ac755d5f8033d0abdca3324/connection.go#L394-L409

The simplest fix is to move stream.closeRemoteChannels() before wg.Wait(), but I'm unsure if it causes other race condition errors.