Unable to control or omit HTTP/2 content-length response header

[gwilym]

trafficstars

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a simple script for reproducing the error.

For serving gRPC over HTTP/2, content-length header should be avoided. Its presence causes issues with some HTTP/2 proxying setups that are unaware of (or agnostic towards) gRPC specifics, such as nghttpx: if ngttpx sees content-length in a response header, then it only reads (content-length) bytes and response trailers are lost.

I tried to omit or override content-length response header but it seems to always be present: https://github.com/swoole/swoole-src/blob/3d2def096e3993d36d45832af8578f1167aacef9/ext-src/swoole_http_response.cc#L484

<?php

require_once __DIR__ . '/vendor/autoload.php';

$server = new Swoole\Http\Server("127.0.0.1", 9501, SWOOLE_PROCESS, SWOOLE_SOCK_TCP);
$server->set([
    'open_http2_protocol' => true,
]);

$server->on('request', function ($request, $response) {
    $response->status(200);
    $response->header('content-length', '123');
    $response->end('foo');
});

$server->start();

I also tried $response->write() instead as a workaround, but: Swoole\Http\Response::write(): HTTP2 client does not support HTTP-CHUNK

2. What did you expect to see?

Some way to control or omit content-length header.

3. What did you see instead?

Always content-length: x in responses (where x = Swoole-calculated length of response data)

4. What version of Swoole are you using (show your php --ri swoole)?

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.5.5
Built => Oct 21 2020 00:36:24
coroutine => enabled
kqueue => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.1h  22 Sep 2020
http2 => enabled
pcre => enabled
zlib => 1.2.11
brotli => E16777225/D16777225
mysqlnd => enabled
async_redis => enabled

5. What is your machine environment used (show your uname -a & php -v & gcc -v) ?

Darwin C02CN25XMD6P 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64

PHP 7.4.11 (cli) (built: Oct  1 2020 23:30:54) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.11, Copyright (c), by Zend Technologies

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.0 (clang-1200.0.26.2)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[malacca]

i think this is about "how to send unknown length (or chunked) message throng http/2"

http/2 not support Transfer-Encoding: chunked message , but it can still do, for example: https://www.youtube.com/

it's response header does not include Transfer-Encoding: chunked and Content-Length

Unfortunately, Swoole has not yet implemented this feature

[software-artificer]

👋 going even further with this, I think it could be beneficial to enable such an operation mode of Swoole that produces no headers, apart from mandatory pseudo-headers. WDYT?

[codercms]

Hi, guys! This is really blocking an ability to stream data to the client. I suggest to implement something like writeFrame method. Look at AMPHP - https://github.com/amphp/http-server/blob/master/src/Driver/Http2Driver.php#L503 (writeBufferedData).

And also I think that the bidirectional streaming should be supported too, I mean it should be possible to server to receive streamed data from the client.

[AdeAttwood]

I have been playing with GRPC this weekend and found this a blocker for the streaming request. All requests closed the stream. There are also a few issues / questions on the swoole-src/grpc repo about streaming.

https://github.com/swoole/grpc/issues/27

[doubaokun]

@malacca @codercms @AdeAttwood

Swoole will allow setting custom content-length header in the future version, does this address your issues?

[AdeAttwood]

I have just had a little play with master, and I am still getting "HTTP2 client does not support HTTP-CHUNK" when trying to ->write() to a http connection. I'm starting to think that allowing users to write to a http2 connection is out of the scope of this issue. When handling a request the only option is to use the end function that closes the connection.

I think my the end game would be to have bi directional streaming via http2 kind of like the message event on a web socket server, so we can implement GRPC services although I don't know how realistic that is. I did start to go down the road of using the receive event on the http server but due to it being over http2 I quickly realized that I would have to implement all the http2 frames and hpack and to achieve that.

A good start would be to allow writing to a http2 connection like @codercms suggested.

$server->on('request', function ($request, $response) {
    while (true) {
        $response->write('data');
    }

   $response->end('data');
});

[codercms]

@doubaokun first at all - thanks for the new release! We need full control on HTTP/2 flow to implement GRPC server functionality, there should be a way to read streamed data from client and stream data from the server to a client at the same time. Currently I have a dirty work-around to stream data from the server to a client - https://github.com/makise-co/grpc-server/blob/feature/initial/src/GrpcServer.php#L355

But I can't implement streamed reading from a client, Swoole doesn't have such functionality.