reva
reva copied to clipboard
cs3org
use tusd datastorages
We are currently ... misusing tusd. Misusing because we implemented our own tusd DataStore. This prevents us from using the existing tusd datastore implementations, eg. the filestore or s3store.
We already require calling InitializeUpload via cs3 before being able to send PATCH requests to the dataprovider. To integrate with tusd we should instead use the PreUploadCreateCallback and PreFinishResponseCallback. Actually, only the latter. The dataprovider never receive a POST directly, so the PreUploadCreateCallback is never called. Instead we need to make InitializeUpload call the same steps as the tusd unrouted handler. Then whe can hand off the complete tus process to tusd and wait for it to make the PreFinishResponseCallback call, where we then calculate checksums and create the actual decomposedfs node.
For this to work we cannot use tusds FileInfo.Storage because it is overwritten when calling NewUpload. So we have to put any metadata into the FileInfo.Metadata map. The latter is also persisted as X-Amz-Meta- in s3.
This PoC works and allows uploading directly to a minio s3 bucket ... with hardcoded credentials ... well ... it is a PoC TODO
- [x] make s3 options configurable
- [ ] fix tests
- [x] add key prefix option to be able to store uploads and blobs in same bucket
- [ ] move callbacks to upload package and delete unused code
- [x] added metadata is exposed in HEAD requests, filter it out? eg the dir reveals the full path even for share recipients. this is already a problem on edge
- [x] prefix all incoming metadata in InitiateUpload()
- [x] check if storing metadata in
X-Amz-Meta-is a problem? not if metadata is filtered when it leaves ocis, eg. in the response of HEAD requests - [ ] pass trace id via metadata
- [ ] test performance in kubernetes
Noteworthy changes
We need to return a fileid after initiating an upload. Currently, we roll the new uuid for new files and put it in the uplod metadata. ... The problem: If there are two concurrent requests trying to create the same child they would receive different fileids.
We can no longer store the new nodeid, postprocessing status or any other metadata in the tus upload metadata. After creating the upload we have to use the node metadata itself (which actually makes sense because all bytes have been transferred).
This requires us to already roll the node id and create a file node in InitializeUpload, and mark it as in processing. This in turn causes new files to show up in directory listings even if not all bytes have been transferred, yet.
hmmm ... how does the curren tcode deal with the problem?
- it stores
info.Storage["NodeExists"] = "false" - this causes
initNewNodeto be called - which would then fail with an AlreadyExists errtype when trying to create the parent symlink
we could do the same in InitiateUpload ... using the symlink creation for atomicity instead of having to use a filelock
simple upload needs to initialize the file upload with the filesize
We currently have a way to decide how to initialize the SizeDeferred flag of a tus upload. We cannot use 0 because empty files do exist. This causes the sample upload to fail ...
InitiateFileUploadRequest does not even have a size proerty ...
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.
![update-docs[bot] avatar](https://avatars.githubusercontent.com/in/4233?v=4 "Published by a pub.dev verified publisher"){kind=small}
@micbar so the tests expect only one version to exist when a file is uploaded with the same mtime. but the code now keeps track of every revision:
Scenario: upload the same file twice with the same mtime and a version is available # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:251
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" uploads file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userUploadsFileToWithMtimeUsingTheWebdavApi()
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "1" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
could not find 1 version element(s) in
<?xml version="1.0"?>
<d:multistatus xmlns:s="http://sabredav.org/ns" xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns"><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$88f60b76-eb21-103d-957c-43ed0cdaa8aa%21a0cc9d8f-5794-41e5-9e27-282999ba5df3/</d:href><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$88f60b76-eb21-103d-957c-43ed0cdaa8aa%21a0cc9d8f-5794-41e5-9e27-282999ba5df3/v/a0cc9d8f-5794-41e5-9e27-282999ba5df3.REV.2023-09-19T10:17:51.960713985Z</d:href><d:propstat><d:prop><d:getetag>"b1e23aeabf59d10f5daadc021b988fd5"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$88f60b76-eb21-103d-957c-43ed0cdaa8aa%21a0cc9d8f-5794-41e5-9e27-282999ba5df3/v/a0cc9d8f-5794-41e5-9e27-282999ba5df3.REV.2023-09-19T10:17:51.926984925Z</d:href><d:propstat><d:prop><d:getetag>"a431c21b9be6bd1327e1f06b40c74d0b"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>
Failed asserting that 2 matches expected 1.
Scenario: upload the same file more than twice with the same mtime and only one version is available # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:258
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
And user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" uploads file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userUploadsFileToWithMtimeUsingTheWebdavApi()
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "1" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
could not find 1 version element(s) in
<?xml version="1.0"?>
<d:multistatus xmlns:s="http://sabredav.org/ns" xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns"><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$8911b43e-eb21-103d-957f-43ed0cdaa8aa%214cfb2093-1f4c-485a-bcff-69c601d5d93c/</d:href><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$8911b43e-eb21-103d-957f-43ed0cdaa8aa%214cfb2093-1f4c-485a-bcff-69c601d5d93c/v/4cfb2093-1f4c-485a-bcff-69c601d5d93c.REV.2023-09-19T10:17:52.178843967Z</d:href><d:propstat><d:prop><d:getetag>"9b010889f9a3f345b2d872d6c8abe030"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$8911b43e-eb21-103d-957f-43ed0cdaa8aa%214cfb2093-1f4c-485a-bcff-69c601d5d93c/v/4cfb2093-1f4c-485a-bcff-69c601d5d93c.REV.2023-09-19T10:17:52.144869863Z</d:href><d:propstat><d:prop><d:getetag>"7349ebd02bc0ba490ece6bab4c95ac15"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$8911b43e-eb21-103d-957f-43ed0cdaa8aa%214cfb2093-1f4c-485a-bcff-69c601d5d93c/v/4cfb2093-1f4c-485a-bcff-69c601d5d93c.REV.2023-09-19T10:17:52.108406347Z</d:href><d:propstat><d:prop><d:getetag>"90d1fa707994f4f6a53232b538bb1bd9"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>
Failed asserting that 3 matches expected 1.
Scenario: upload the same file twice with the same mtime and no version after restoring # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:266
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
And user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" restores version index "1" of file "/file.txt" using the WebDAV API # FilesVersionsContext::userRestoresVersionIndexOfFile()
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "0" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
could not find 0 version element(s) in
<?xml version="1.0"?>
<d:multistatus xmlns:s="http://sabredav.org/ns" xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns"><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$89344148-eb21-103d-9582-43ed0cdaa8aa%21546c80fa-3ed2-4cea-ae9c-0e33a552c7b2/</d:href><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$89344148-eb21-103d-9582-43ed0cdaa8aa%21546c80fa-3ed2-4cea-ae9c-0e33a552c7b2/v/546c80fa-3ed2-4cea-ae9c-0e33a552c7b2.REV.2023-09-19T10:17:52.333974461Z</d:href><d:propstat><d:prop><d:getetag>"e32940ea5bae7aa1271e92cf6d38fde2"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$89344148-eb21-103d-9582-43ed0cdaa8aa%21546c80fa-3ed2-4cea-ae9c-0e33a552c7b2/v/546c80fa-3ed2-4cea-ae9c-0e33a552c7b2.REV.2019-08-08T04:18:13Z</d:href><d:propstat><d:prop><d:getetag>"b2516215c4b4930ea43a608875ac06f4"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>
Failed asserting that 2 matches expected 0.
I'm not sure if we want this keep this arguably broken revision tracking?
Thesize diff calculation might be racy:
- there is an existing file with size 10
- upload A is started and writes a 5 byte file - this will set the file size to 5 when all bytes are uploaded
- upload B is started and writes a 15 byte file - this will set the file size to 15 when all bytes are uploaded
- all bytes for B arrived. size is 15 postprocessing B is started
- all bytes for A arrived. size is 5 postprocessing A is started
- postprocessing B fails, so the sizediff of 15 is subtracted ... an the file now has size -10?
postprocessing B fails, so the sizediff of 15 is subtracted ... an the file now has size -10?
No, the result is 15
10 + 5 + 15 -15 = 15
so the error seems to be that the revision has no blobid attribute set ...
{"level":"error","error":"error reading blobid xattr: xattr.get /drone/src/tmp/reva/data/spaces/28/175508-ec25-103d-820e-9b9f4a005ed7/nodes/bd/1b/06/73/-b87d-49f8-bcab-535b42cc37dc.REV.2023-09-20T17:16:18.608228057Z.mlock user.ocis.blobid: no data available","revision":"/drone/src/tmp/reva/data/spaces/28/175508-ec25-103d-820e-9b9f4a005ed7/nodes/bd/1b/06/73/-b87d-49f8-bcab-535b42cc37dc.REV.2023-09-20T17:16:18.608228057Z.mlock","time":"2023-09-20T17:16:26.941046467Z","message":"error reading blobid attribute"}
[90m2023-09-20 17:16:26.941[0m [1m[31mERR[0m[0m [1m../../../internal/grpc/services/storageprovider/storageprovider.go:981[0m[36m >[0m failed to empty recycle [36merror=[0m[31m"error reading blobid xattr: xattr.get /drone/src/tmp/reva/data/spaces/28/175508-ec25-103d-820e-9b9f4a005ed7/nodes/bd/1b/06/73/-b87d-49f8-bcab-535b42cc37dc.REV.2023-09-20T17:16:18.608228057Z.mlock user.ocis.blobid: no data available"[0m [36mkey=[0m/ [36mpid=[0m12 [36mpkg=[0mrgrpc [36mreference=[0m{"path":".","resource_id":{"opaque_id":"28175508-ec25-103d-820e-9b9f4a005ed7","space_id":"28175508-ec25-103d-820e-9b9f4a005ed7","storage_id":"1284d238-aa92-42ce-bdc4-0b0000009157"}} [36mstatus=[0m{"code":15,"message":"error emptying recycle:error reading blobid xattr: xattr.get /drone/src/tmp/reva/data/spaces/28/175508-ec25-103d-820e-9b9f4a005ed7/nodes/bd/1b/06/73/-b87d-49f8-bcab-535b42cc37dc.REV.2023-09-20T17:16:18.608228057Z.mlock user.ocis.blobid: no data available","trace":"038deb2cd375cbe1eccb9e2e361a3c9b"} [36mtraceid=[0ma675d2a0acf71398263689c8bda6c54c
[90m2023-09-20 17:16:26.941[0m [33mDBG[0m [1m../../../internal/grpc/interceptors/log/log.go:69[0m[36m >[0m unary [36mcode=[0mOK [36mend=[0m"20/Sep/2023:17:16:26 +0000" [36mfrom=[0mtcp://127.0.0.1:53086 [36mpid=[0m12 [36mpkg=[0mrgrpc [36mstart=[0m"20/Sep/2023:17:16:26 +0000" [36mtime_ns=[0m10453361 [36mtraceid=[0ma675d2a0acf71398263689c8bda6c54c [36muri=[0m/cs3.storage.provider.v1beta1.ProviderAPI/PurgeRecycle [36muser-agent=[0mGuzzleHttp/7
[90m2023-09-20 17:16:26.941[0m [33mDBG[0m [1m../../../internal/grpc/interceptors/log/log.go:69[0m[36m >[0m unary [36mcode=[0mOK [36mend=[0m"20/Sep/2023:17:16:26 +0000" [36mfrom=[0mtcp://127.0.0.1:44532 [36mpid=[0m10 [36mpkg=[0mrgrpc [36mstart=[0m"20/Sep/2023:17:16:26 +0000" [36mtime_ns=[0m11953954 [36mtraceid=[0m3e1880e0396e5774c6c5ad2c2f55a4fd [36muri=[0m/cs3.gateway.v1beta1.GatewayAPI/PurgeRecycle [36muser-agent=[0mGuzzleHttp/7
[90m2023-09-20 17:16:26.942[0m [1m[31mERR[0m[0m [1m../../../internal/http/services/owncloud/ocdav/errors/error.go:174[0m[36m >[0m Internal Server Error [36mcode=[0m500 [36mitem_path=[0m/ [36mkey=[0m [36mpid=[0m9 [36mpkg=[0mrhttp [36mreference=[0m{"path":".","resource_id":{"opaque_id":"28175508-ec25-103d-820e-9b9f4a005ed7","space_id":"28175508-ec25-103d-820e-9b9f4a005ed7","storage_id":"1284d238-aa92-42ce-bdc4-0b0000009157"}} [36mstatus=[0m{"code":15,"message":"error emptying recycle:error reading blobid xattr: xattr.get /drone/src/tmp/reva/data/spaces/28/175508-ec25-103d-820e-9b9f4a005ed7/nodes/bd/1b/06/73/-b87d-49f8-bcab-535b42cc37dc.REV.2023-09-20T17:16:18.608228057Z.mlock user.ocis.blobid: no data available","trace":"038deb2cd375cbe1eccb9e2e361a3c9b"} [36mtraceid=[0m038deb2cd375cbe1eccb9e2e361a3c9b
[90m2023-09-20 17:16:26.942[0m [1m[31mERR[0m[0m [1m../../../internal/http/interceptors/log/log.go:112[0m[36m >[0m http [36mend=[0m"20/Sep/2023:17:16:26 +0000" [36mhost=[0m192.168.29.5 [36mmethod=[0mDELETE [36mpid=[0m9 [36mpkg=[0mrhttp [36mproto=[0mHTTP/1.1 [36msize=[0m0 [36mstart=[0m"20/Sep/2023:17:16:26 +0000" [36mstatus=[0m500 [36mtime_ns=[0m29996616 [36mtraceid=[0m038deb2cd375cbe1eccb9e2e361a3c9b [36muri=[0m/remote.php/dav/trash-bin/Alice [36murl=[0m/
- [ ] we should move the code to get the list of uploads from decompesedfs to a tus package ... uploads are not storagedriver specific, but tus datastore specific. we might contribute them upstream if they want them.
I rebased on top of latest reva. no big difference, looking at the current errors.
I don't know what is going on with the S3NG driver. IT does not seem to change the checksum ... in the log all cheksums are the same:
2023-11-01 13:40:45.882 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=091bc15c-f6d4-42fb-841e-d9e8d468431a pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"1112d204aeb097963bb7355bd420ec19\"","id":{"opaque_id":"091bc15c-f6d4-42fb-841e-d9e8d468431a","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":237217658,"seconds":1698846039},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"88934515-d122-4682-b09c-69253e277d3c","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=16a577dc4a5dfad5934a582c4f03c6d8
2023-11-01 13:40:46.39 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=4123baa3-aad8-4a65-8583-cc51b29c67ae pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"8646f95ab819510f114ec92492ae34cc\"","id":{"opaque_id":"4123baa3-aad8-4a65-8583-cc51b29c67ae","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":983751921,"seconds":1698846045},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"c47dce33-4955-415c-ae8a-398ad423e36e","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=21a0e077cfa1b71aa48fe53b147175ea
2023-11-01 13:40:46.921 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cd5b25f4-49e7-4318-9bb7-3c3523073646 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"d31836684af382b6ccb39310494b7b51\"","id":{"opaque_id":"cd5b25f4-49e7-4318-9bb7-3c3523073646","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":508119371,"seconds":1698846046},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"a461a9a3-edfc-43a8-bb5b-2ea198921f46","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=2f98d91cc260627d96f20b35c726f832
2023-11-01 13:40:46.956 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cd5b25f4-49e7-4318-9bb7-3c3523073646 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"d31836684af382b6ccb39310494b7b51\"","id":{"opaque_id":"cd5b25f4-49e7-4318-9bb7-3c3523073646","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":508119371,"seconds":1698846046},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"a461a9a3-edfc-43a8-bb5b-2ea198921f46","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=0073648fc16c532e76b63bd839e14249
2023-11-01 13:40:47.457 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=269512c8-fc14-403a-90a7-80ea27e5acb9 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"c6b07d58f65560996be6104130000cd1\"","id":{"opaque_id":"269512c8-fc14-403a-90a7-80ea27e5acb9","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":56569403,"seconds":1698846047},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"0dee8e04-c092-422d-a9ee-84e27a167e5c","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=4e739bd3e6f86fdeb04ac1d433d8bd8e
2023-11-01 13:40:47.933 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=b3bcba28-1e94-4b05-9b47-79f61a8ace07 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"50f41d31401bec2f8194e15e871c00ae\"","id":{"opaque_id":"b3bcba28-1e94-4b05-9b47-79f61a8ace07","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":471408166,"seconds":1698846047},"name":"testfile2.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"0dee8e04-c092-422d-a9ee-84e27a167e5c","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile2.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=38da573821a8243502f25f5e4a75ff7b
2023-11-01 13:40:48.417 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=705c459d-ccb6-44d1-bd3f-db5d97f6f5e4 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"4aa0ca37b7bfa01b8afb0f9c27ffcc29\"","id":{"opaque_id":"705c459d-ccb6-44d1-bd3f-db5d97f6f5e4","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":11574237,"seconds":1698846048},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"32859e68-58d1-47a9-aca5-f276b683a418","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=84b47d669f9d2fe0382cff1220fe8940
2023-11-01 13:40:48.885 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=9f0b42ea-2c07-4050-b0d8-8ba42d1b0db5 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"aa0dc1e084ca9527ebd086e4d2e03205\"","id":{"opaque_id":"9f0b42ea-2c07-4050-b0d8-8ba42d1b0db5","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":431815619,"seconds":1698846048},"name":"testfile2.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"20ab60f8-fe41-4726-a6e3-dac22683c1ab","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile2.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=df112e33d2e066628951bae7f13c065a
2023-11-01 13:40:49.365 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"f682bba2167bfa6798bdbdcc079d3f96\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":965061958,"seconds":1698846048},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=ff3d93a60eef7a0160d0cc2edd3c8d4c
2023-11-01 13:40:49.368 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"f682bba2167bfa6798bdbdcc079d3f96\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":965061958,"seconds":1698846048},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=6031ad50ca49f982bc19113d9e222e8d
2023-11-01 13:40:49.377 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"f682bba2167bfa6798bdbdcc079d3f96\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":965061958,"seconds":1698846048},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=d76c0244e3bcb631f5f82c4c8e20d601
2023-11-01 13:40:49.38 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"f682bba2167bfa6798bdbdcc079d3f96\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":965061958,"seconds":1698846048},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":4,"type":1} traceid=d001a07f2e78c477ab93ef971b963b05
2023-11-01 13:40:49.822 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"b408cf3de57cd61ecb815fb51b1a9876\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":381622079,"seconds":1698846049},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":8,"type":1} traceid=1f26f82f40ecb8d764fe86c86b93dbaf
2023-11-01 13:40:49.826 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=cfeb456b-d0a9-4c10-81a9-f966b1c9de51 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"b408cf3de57cd61ecb815fb51b1a9876\"","id":{"opaque_id":"cfeb456b-d0a9-4c10-81a9-f966b1c9de51","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":381622079,"seconds":1698846049},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"6b890791-0f16-436b-a34f-52d1d8b44d00","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":8,"type":1} traceid=065bddf142c59768e31cc32b926581db
2023-11-01 13:40:50.293 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=45c54439-7432-4f6d-bfab-74ed7c87c3db pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"2ed0a9ef485e09fa89a62701562a11c2\"","id":{"opaque_id":"45c54439-7432-4f6d-bfab-74ed7c87c3db","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":886834954,"seconds":1698846049},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=7accb4fa50787572b4c07ef911cb7ba8
2023-11-01 13:40:50.296 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=45c54439-7432-4f6d-bfab-74ed7c87c3db pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"2ed0a9ef485e09fa89a62701562a11c2\"","id":{"opaque_id":"45c54439-7432-4f6d-bfab-74ed7c87c3db","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":886834954,"seconds":1698846049},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=330e875e4c51f0f5fe8c9894a38b9e0f
2023-11-01 13:40:50.304 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=45c54439-7432-4f6d-bfab-74ed7c87c3db pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"2ed0a9ef485e09fa89a62701562a11c2\"","id":{"opaque_id":"45c54439-7432-4f6d-bfab-74ed7c87c3db","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":886834954,"seconds":1698846049},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=d0ce5537c920a4bf2c78be0fdbb26bf0
2023-11-01 13:40:50.769 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=113e1828-4f71-4f93-acee-a973e40bb42b pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"f3e2b5268d351aeefe9a2ab61c5bdcec\"","id":{"opaque_id":"113e1828-4f71-4f93-acee-a973e40bb42b","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":369177028,"seconds":1698846050},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"82524ddc-0490-4585-9aa4-fa266af7929c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=c8b90e7ee8e305c5a6546ce985999bcd
2023-11-01 13:40:51.237 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=92c89c24-f949-4b7b-8c53-f705610edbe8 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"5e3e055a09498a46cc4a3c607955a99a\"","id":{"opaque_id":"92c89c24-f949-4b7b-8c53-f705610edbe8","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":834508195,"seconds":1698846050},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=1d3330c8dcafb1f3c0cc2b7ef5a32bfd
2023-11-01 13:40:51.24 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=92c89c24-f949-4b7b-8c53-f705610edbe8 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"5e3e055a09498a46cc4a3c607955a99a\"","id":{"opaque_id":"92c89c24-f949-4b7b-8c53-f705610edbe8","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":834508195,"seconds":1698846050},"name":"testfile.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"testfile.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=fe992a99fa98e538ad33187f91b14774
2023-11-01 13:40:51.258 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=92c89c24-f949-4b7b-8c53-f705610edbe8 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"5e3e055a09498a46cc4a3c607955a99a\"","id":{"opaque_id":"92c89c24-f949-4b7b-8c53-f705610edbe8","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":834508195,"seconds":1698846050},"name":"Test-File.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"Test-File.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=536b2b4bcc6c0a43906d3d325f9b9bf6
2023-11-01 13:40:51.26 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=92c89c24-f949-4b7b-8c53-f705610edbe8 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"5e3e055a09498a46cc4a3c607955a99a\"","id":{"opaque_id":"92c89c24-f949-4b7b-8c53-f705610edbe8","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":834508195,"seconds":1698846050},"name":"Test-File.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"Test-File.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=ccb80074b528c5a3011998b40ca01b2e
2023-11-01 13:40:51.27 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=92c89c24-f949-4b7b-8c53-f705610edbe8 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"5e3e055a09498a46cc4a3c607955a99a\"","id":{"opaque_id":"92c89c24-f949-4b7b-8c53-f705610edbe8","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"mime_type":"text/plain","mtime":{"nanos":834508195,"seconds":1698846050},"name":"Test-File.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","type":1},"parent_id":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","space_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"path":"Test-File.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"size":11,"type":1} traceid=88ffc6f81f3a39362f5eca32d173415f
2023-11-01 13:40:51.868 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=f4ea75a0-c09b-440e-a56f-57f11996f468 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"635d826ac686d62df2da532257813ed8\"","id":{"opaque_id":"f4ea75a0-c09b-440e-a56f-57f11996f468","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":419090839,"seconds":1698846051},"name":"test.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"129cdf0d-de72-463e-8ef6-5ade7582391e","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"/public-office-documents/test.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"type":1} traceid=8ad8dab675f3667d6586476f084bc77e
2023-11-01 13:40:52.376 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=f4ea75a0-c09b-440e-a56f-57f11996f468 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"635d826ac686d62df2da532257813ed8\"","id":{"opaque_id":"f4ea75a0-c09b-440e-a56f-57f11996f468","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":419090839,"seconds":1698846051},"name":"test.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"129cdf0d-de72-463e-8ef6-5ade7582391e","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"/public-office-documents/test.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"type":1} traceid=ae3a962c9990e05460388bcf1e269589
2023-11-01 13:40:52.376 DBG ../../../pkg/storage/utils/decomposedfs/node/node.go:836 > AsResourceInfo node=d5a1aaa5-4f8a-4ee1-9c9d-bf6eabd41d70 pid=11 pkg=rgrpc ri={"arbitrary_metadata":{"metadata":{"http://owncloud.org/ns/favorite":""}},"checksum":{"sum":"da39a3ee5e6b4b0d3255bfef95601890afd80709","type":4},"etag":"\"9943e40c65cd0380ea7526ee77d79d81\"","id":{"opaque_id":"d5a1aaa5-4f8a-4ee1-9c9d-bf6eabd41d70","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"mime_type":"text/plain","mtime":{"nanos":892700322,"seconds":1698846051},"name":"test2.txt","opaque":{"map":{"adler32":{"decoder":"plain","value":"MDAwMDAwMDE="},"md5":{"decoder":"plain","value":"ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U="}}},"owner":{"idp":"http://localhost:20080","opaque_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","type":1},"parent_id":{"opaque_id":"129cdf0d-de72-463e-8ef6-5ade7582391e","space_id":"ddc2004c-0977-11eb-9d3f-a793888cd0f8"},"path":"/public-office-documents/test2.txt","permission_set":{"add_grant":true,"create_container":true,"delete":true,"deny_grant":true,"get_path":true,"get_quota":true,"initiate_file_download":true,"initiate_file_upload":true,"list_container":true,"list_file_versions":true,"list_grants":true,"list_recycle":true,"move":true,"purge_recycle":true,"remove_grant":true,"restore_file_version":true,"restore_recycle_item":true,"stat":true,"update_grant":true},"type":1} traceid=ae3a962c9990e05460388bcf1e269589
must have something to do with the s3 backend ... the tests use the xattr metadata_backend ... will check that :eyes:
da39a3ee5e6b4b0d3255bfef95601890afd80709 is the hash for the empty string ... maybe reading back the upload from ceph does not work?
Indeed. The added log line appears: internal error: mismatching upload length. expected 4, could only copy 0
👀
2023-11-01 15:13:14.018 DBG ../../../pkg/rhttp/rhttp.go:248 > http routing: head=data tail=/simple/ef186ae941dd9250bf46163afe2772b6+2~PxXm3ze5qcW3AfYzPWkgfjAmKQmrtNG svc=data pid=11 pkg=rhttp
2023-11-01 15:13:14.018 DBG ../../../internal/http/services/dataprovider/dataprovider.go:179 > dataprovider routing: path=/simple/ef186ae941dd9250bf46163afe2772b6+2~PxXm3ze5qcW3AfYzPWkgfjAmKQmrtNG pid=11 pkg=rhttp traceid=af187840ef124bd7d3ae7577f64b1218
2023-11-01 15:13:14.381 ERR ../../../pkg/rhttp/datatx/manager/simple/simple.go:140 > error uploading file error="internal error: mismatching upload length. expected 4, could only copy 0" datatx=simple pid=11 pkg=rhttp traceid=af187840ef124bd7d3ae7577f64b1218
2023-11-01 15:13:14.381 ERR ../../../internal/http/interceptors/log/log.go:112 > http end="01/Nov/2023:15:13:14 +0000" host=192.168.3.4 method=PUT pid=11 pkg=rhttp proto=HTTP/1.1 size=0 start="01/Nov/2023:15:13:14 +0000" status=500 time_ns=362901001 traceid=af187840ef124bd7d3ae7577f64b1218 uri=/data/simple/ef186ae941dd9250bf46163afe2772b6+2~PxXm3ze5qcW3AfYzPWkgfjAmKQmrtNG url=/ef186ae941dd9250bf46163afe2772b6+2~PxXm3ze5qcW3AfYzPWkgfjAmKQmrtNG
🤔 why are the uploads done via /simple/{uploadid} and not /tus/{uploadid} ... hm ... might be the cs3 validator ... locally the tus upload via web ui works fine ... and it uses /tus/{uploadid}
👀
the cs3 validation trests do not send the size:
2023-11-02 11:46:45.461 DBG ../../../pkg/storage/utils/decomposedfs/upload.go:235 > Decomposedfs: initiated upload headers={"providerID":"1284d238-aa92-42ce-bdc4-0b0000009157"} info={"ID":"06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73","IsFinal":false,"IsPartial":false,"MetaData":{"cs3.ExecutantId":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.ExecutantIdp":"http://localhost:20080","cs3.ExecutantType":"primary","cs3.ExecutantUserName":"admin","cs3.LogLevel":"debug","cs3.NodeId":"","cs3.NodeParentId":"5dd5fd83-7baf-4f62-9812-a213249d994a","cs3.RevisionTime":"2023-11-02T11:46:45.451162592Z","cs3.SpaceOwnerOrManager":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.SpaceRoot":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.dir":"/a-folder/a-sub-folder","cs3.filename":"testfile.txt","cs3.lockid":"","cs3.providerID":"1284d238-aa92-42ce-bdc4-0b0000009157"},"Offset":0,"PartialUploads":null,"Size":0,"SizeIsDeferred":true,"Storage":{"Bucket":"test","Key":"06a666a5d9531506c1ed5ab1d96cba02","Type":"s3store"}} nodeid= pid=11 pkg=rgrpc spaceid=ddc2004c-0977-11eb-9d3f-a793888cd0f8 traceid=bd2759a62c350e8ff677f1266e7c6235 uploadLength=0
2023-11-02 11:46:45.461 INF ../../../internal/grpc/services/storageprovider/storageprovider.go:455 > file upload data-server=http://revad-services:11001/data/simple/06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73 fn=./a-folder/a-sub-folder/testfile.txt pid=11 pkg=rgrpc traceid=bd2759a62c350e8ff677f1266e7c6235 xs="map[md5:100 unset:1000]"
2023-11-02 11:46:45.461 INF ../../../internal/grpc/services/storageprovider/storageprovider.go:455 > file upload data-server=http://revad-services:11001/data/tus/06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73 fn=./a-folder/a-sub-folder/testfile.txt pid=11 pkg=rgrpc traceid=bd2759a62c350e8ff677f1266e7c6235 xs="map[md5:100 unset:1000]"
2023-11-02 11:46:45.462 DBG ../../../internal/grpc/interceptors/log/log.go:69 > unary code=OK end="02/Nov/2023:11:46:45 +0000" from=tcp://127.0.0.1:48388 pid=11 pkg=rgrpc start="02/Nov/2023:11:46:45 +0000" time_ns=11587125 traceid=bd2759a62c350e8ff677f1266e7c6235 uri=/cs3.storage.provider.v1beta1.ProviderAPI/InitiateFileUpload user-agent=grpc-go/1.56.2
2023-11-02 11:46:45.462 DBG ../../../internal/grpc/interceptors/log/log.go:69 > unary code=OK end="02/Nov/2023:11:46:45 +0000" from=tcp://192.168.44.2:46340 pid=10 pkg=rgrpc start="02/Nov/2023:11:46:45 +0000" time_ns=12718409 traceid=7952a4c39bb0929843873e2f34099aba uri=/cs3.gateway.v1beta1.GatewayAPI/InitiateFileUpload user-agent=grpc-go/1.42.0
2023-11-02 11:46:45.463 DBG ../../../pkg/rhttp/rhttp.go:248 > http routing: head=data tail=/simple/06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73 svc=data pid=11 pkg=rhttp
2023-11-02 11:46:45.463 DBG ../../../internal/http/services/dataprovider/dataprovider.go:179 > dataprovider routing: path=/simple/06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73 pid=11 pkg=rhttp traceid=2e2c7ac0bd6c3189c0455fe3562bb671
{"level":"error","pid":11,"info":{"ID":"06a666a5d9531506c1ed5ab1d96cba02+2~xksXoAaOlflsXkAhUDEM5NsVKJ2iD73","Size":4,"SizeIsDeferred":true,"Offset":0,"MetaData":{"cs3.ExecutantId":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.ExecutantIdp":"http://localhost:20080","cs3.ExecutantType":"primary","cs3.ExecutantUserName":"admin","cs3.LogLevel":"debug","cs3.NodeId":"","cs3.NodeParentId":"5dd5fd83-7baf-4f62-9812-a213249d994a","cs3.RevisionTime":"2023-11-02T11:46:45.451162592Z","cs3.SpaceOwnerOrManager":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.SpaceRoot":"ddc2004c-0977-11eb-9d3f-a793888cd0f8","cs3.dir":"/a-folder/a-sub-folder","cs3.filename":"testfile.txt","cs3.lockid":"","cs3.providerID":"1284d238-aa92-42ce-bdc4-0b0000009157"},"IsPartial":false,"IsFinal":false,"PartialUploads":null,"Storage":{"Bucket":"test","Key":"06a666a5d9531506c1ed5ab1d96cba02","Type":"s3store"}},"time":"2023-11-02T11:46:45.798710683Z","caller":"/drone/src/pkg/storage/utils/decomposedfs/upload.go:309","message":"mismatching upload length. expected 4, could only copy 0"}
2023-11-02 11:46:45.798 ERR ../../../pkg/rhttp/datatx/manager/simple/simple.go:140 > error uploading file error="internal error: mismatching upload length. expected 4, could only copy 0" datatx=simple pid=11 pkg=rhttp traceid=2e2c7ac0bd6c3189c0455fe3562bb671
maybe I forgot some code in the simple handler ... 👀 urgh ... the InitiateFileUploadRequest does not even have a size property 🤦♂️
resp, err := f.Client.InitiateFileUpload(
ctx,
&providerv1beta1.InitiateFileUploadRequest{
Ref: &providerv1beta1.Reference{
//ResourceId: publicShareResp.Share.ResourceId,
Path: path,
},
},
)
@micbar again ... messing with versions. The tests are writing revisions with specific mtimes, multiple times:
Scenario: upload the same file twice with the same mtime and a version is available # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:251
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" uploads file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userUploadsFileToWithMtimeUsingTheWebdavApi()
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "1" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
could not find 1 version element(s) in
<?xml version="1.0"?>
<d:multistatus xmlns:s="http://sabredav.org/ns" xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns"><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$fd5645c0-1683-103e-97a4-6fb7d9ca50a3%211bfcdae7-9716-4d77-82d2-70615096d64e/</d:href><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response></d:multistatus>
Failed asserting that 0 matches expected 1.
Scenario: upload the same file more than twice with the same mtime and only one version is available # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:258
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
And user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" uploads file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userUploadsFileToWithMtimeUsingTheWebdavApi()
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "1" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
could not find 1 version element(s) in
<?xml version="1.0"?>
<d:multistatus xmlns:s="http://sabredav.org/ns" xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns"><d:response><d:href>/remote.php/dav/meta/1284d238-aa92-42ce-bdc4-0b0000009157$fd73aa2a-1683-103e-97a7-6fb7d9ca50a3%21a3ffc4fb-399e-49f7-b01e-54141084202a/</d:href><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response></d:multistatus>
Failed asserting that 0 matches expected 1.
Scenario: upload the same file twice with the same mtime and no version after restoring # /drone/src/tmp/testrunner/tests/acceptance/features/coreApiVersions/fileVersions.feature:266
Given user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
And user "Alice" has uploaded file "filesForUpload/textfile.txt" to "file.txt" with mtime "Thu, 08 Aug 2019 04:18:13 GMT" using the WebDAV API # FeatureContext::userHasUploadedFileToWithMtimeUsingTheWebdavApi()
When user "Alice" restores version index "1" of file "/file.txt" using the WebDAV API # FilesVersionsContext::userRestoresVersionIndexOfFile()
Notice: Undefined offset: 1 in /drone/src/tmp/testrunner/tests/acceptance/features/bootstrap/FilesVersionsContext.php line 154
Then the HTTP status code should be "204" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
And the version folder of file "/file.txt" for user "Alice" should contain "0" element # FilesVersionsContext::theVersionFolderOfFileShouldContainElements()
now ... I am keeping track of every upload as revision. So the first upload already creates a new revision. I have to hide it or a lot of other tests fail because they don't expect a revision to exist when the file has only been written once ...
🤔 maybe I can omit creating tho .REV. metadata file ...
I am going in circles. The old code would write a file into the /uploads folder of a space. Only when the upload was complete it would create a revision file.
To reuse the tus datastore I initially chose to store the upload state for processing in a dedicated file by creating the revision when the upload started. That way I don't have to copy around metadata later. But the revisions that are being uploaded get picked up by the list Revisions call. So they have to be filtered out.
Another problem is that the x-oc-mtime header is used in tests to write the same file with the same mtime multiple times. To mimic the oc10 versions that requires writing the node metadata and replacing the revision metadata.
... 🤔 ...
I was thinking of keeping track about upload metadata in {nodeid}.UPLOAD.{revisiontime}.mpk files, but I think it makes more sense to reuse the /uploads folder and only store our metadata there. We need to be able to list ongoing uploads anyway. The difference would be that we only store metadata there, no blobs. unless the filestore of tus is used.
Ok but this allows us to solve both problems with one solution and correctly implement the revisions API.
I'm back to tha cave!
hm lots of failures when testing against ocis: https://drone.owncloud.com/owncloud/ocis/29114/28/6
We evolved our own DataStore into a OcisSession that also covers postprocessing.