Direct upload gets stuck in dev mode

[agazso]

Context

Bee 1.7, macOS

Summary

I was working on an app that uses direct upload (== deferred: false) for uploading chunks. The app is working with a Bee node running on mainnet, however as soon as I try it in dev mode it gets stuck at the first time it tries to upload a chunk. If the same chunk was already uploaded in deferred mode the upload will succeed though.

Expected behavior

Either the direct upload should work or it should return an error indicating that the direct upload is not supported in dev mode.

Steps to reproduce

1. Start Bee in dev mode (e.g. ./bee dev)
2. Buy a postage stamp (swarm-cli stamp buy --depth 20 --amount 1000000)
3. Store the postage batch id in an environment variable called STAMP (export STAMP="your-postage-batch-id")
4. Generate empty chunk (dd if=/dev/zero of=./empty count=8 bs=1)
5. Upload the empty file with direct mode:

curl \
    -X POST \
    -H "Swarm-Deferred-Upload: false" \
    -H "Swarm-Postage-Batch-Id: $STAMP" \
    --data-binary @empty http://localhost:1633/chunks

The last step will get stuck forever with Bee printing nothing in the logs. If you use the same command but change the deferred upload to true then it will be succesful (and after that direct uploads will succeed as well):

 % curl \
    -X POST \
    -H "Swarm-Deferred-Upload: true" \ 
    -H "Swarm-Postage-Batch-Id: $STAMP" \
    --data-binary @empty http://localhost:1633/chunks

{"reference":"b34ca8c22b9e982354f9c7f50b470d66db428d880c8a904d5fe4ec9713171526"}

[ldeffenb]

I'd be curious what a pre-created tag would show as the deferred upload actually gets pushed, especially the debug API query of the tag after the deferred upload happens. The debug query of the tag provides more details into what's happening inside the bee node.

Also, once chunks are known to the node, uploading them again is pretty much a no-op which would explain the direct upload working after a deferred upload has locally stored the chunks and put them in the pusher queue.

[mrekucci]

If the problem is the same as described in #2946, it can be "fixed" as in #3041.

[agazso]

I am not sure that the #3401 can be considered a proper fix for this issue. If I understand correctly that makes it timeout and fail with an error saying idle read timeout exceeded.

If there is a fundamental reason why the direct upload should not work in dev mode then the proper fix would be to return HTTP 400 Bad request and the error message could tell that direct upload is not supported in dev mode.