opencloud icon indicating copy to clipboard operation
opencloud copied to clipboard

enhancement(proxy): add read deadline and filter chain middleware

Open fschade opened this issue 1 month ago • 0 comments

Description

as described here openCloud does not close dangling uploads which can result in unnecessary unclosed connections.

Usually openCloud is used behind a proxy such as nginx which has timout settings for such cases, e.g.:

  • proxy_connect_timeout
  • proxy_send_timeout
  • proxy_read_timeout
  • send_timeout

which works and fixes that!

Anyway in my opinion openCloud should also have such options just in case.

As a starting-point the PR introduces such tools and already fixes it for the described upload case here.

The difference to the nginx timeout is that this implementation refreshes the timeout after each read of the request body which is specially helpful for large uploads with probably many chunks!

In a nutshell it contains the following tools:

  • a middleware that allows to inspect and apply middleware chains per request
  • a middleware that sets a deadline for uploads and refreshes that after each read cycle

unfortunately the last uploaded chunk could take some time to finish because we have to calculate the checksum here and if the file is large the calculation will take some time, thats the reason behind the long upload deadline default value (120s).

as discussed today @butonic and @rhafer we maybe can send the checksum calculation to the background and not as part of the last upload chunk request! NOT PART OF THIS PR!

Related Issue

  • Fixes https://github.com/opencloud-eu/opencloud/issues/1643

Motivation and Context

close unused or dangling requests

How Has This Been Tested?

#!/usr/bin/env bash

echo 1
LOCATION=$(curl -k -I -s -o /dev/null -w '%header{location}' -XPOST 'https://localhost:9200/remote.php/dav/spaces/281457a5-5279-47d7-b55e-9f514c2df42b%24b6b9f660-e084-4dda-9689-76b75777381e' -uadmin:admin -H"Tus-Resumable: 1.0.0" -H"Upload-Length: 10" -H"Upload-Metadata: filename ZmlsZS50eHQ=" -H"Content-Type: application/offset+octet-stream")
echo 2
echo "$LOCATION"
echo "========================="
echo `date`: PATCH start
echo "========================="
curl -vk -XPATCH "$LOCATION" -H'Content-Type: application/offset+octet-stream' -H'Content-Length: 10' -H'Upload-Offset: 0' -H'Tus-Resumable: 1.0.0' -d"012345678"
echo "========================="
echo `date`: PATCH end
echo "========================="
echo 3

Types of changes

  • [X] Bug fix (non-breaking change which fixes an issue)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [X] Technical debt
  • [ ] Tests only (no source changes)

Checklist:

  • [X] Code changes
  • [X] Unit tests added
  • [ ] Acceptance tests added
  • [ ] Documentation added

fschade avatar Nov 20 '25 12:11 fschade