bee
bee copied to clipboard
ethersphere
Bee may return 200 in case of missing data
Summary
It is possible that Bee returns HTTP 200 OK as a response in case of data is missing when using the /bytes or the /bzz endpoints. This is because the underlying implementation uses a lookahead buffer and returns data lazily, starting with returning a 200 header, even the data is not found or incomplete later.
Expected behavior
The expected behavior is that Bee returns 4XX HTTP error, indicating that it wasn't able to find all chunks during retrieval.
Possible solution
Unfortunately this use-case is not trivial to support with HTTP/1.1. There is chunked encoding support but that does not let the status to be changed after the chunk streaming started, only adding additional headers. It would be possible to add a custom header indicating that there was a problem, but that is very ad-hoc and clients would need to additionally implement that.
Another option is to limit the size of the response and if that does not arrive in time return a 4XX error. This use-case may be better supported in HTTP/2 and maybe it above the size limit only HTTP/2 requests should be allowed. Using websockets for supporting bigger datasets may be another option.
@agazso Could you please in the meantime add a custom header as a workaround? I know it is not a solution, but I'm trying to fix with a workaround this issue, because the CDN caches the empty response.
I'm using Nginx, and the only solution that I found at moment is to look for headers. proxy_pass doesn't expose the response body length. Unluckily with the chunk encoding I don't have a Content-Length header, so I can't check if it is 0. An error header would solve for me. At least for the workaround.
Or, for me, a simple Cache-Control: no-cache could be already a solution. And it's a standard header.
I've talked with CDN assistance, and also they can't handle this case. I can't solve in other ways than fixing on Bee side.
This can happen even if content should be pinned. I've tried to pin an already existing content, pin succeeded and it's reported on pinned contents list. Despite this, every time that I try to request it the node reply code 200 with empty page.
When you re-pin an existing pin from the API, nothing actually happens in the bee node. It just returns success. But if chunks have been unpinned below that reference, possibly by #3037 , then subsequent retrievals can actually still fail to retrieve supposedly pinned references.
I didn't re-pin. My "already existing content" was intended on the net. I wasn't pinning on upload, but with a POST on /pins/<resource>. Anyway, content should be pinned, and instead returns empty page.
We recently fixed the issue where the node now correctly sends the content length of the data. it is not possible to send a 200 status without sending the data. With the content length, for example, browsers and other clients are able to detect the error if fewer amount of data than the content length was received.
