Chocobozzz
create-move-video-storage-job fails to download video
Describe the current behavior
Currently, when trying to use create-move-video-storage-job to download a S3 stored video the video doesn't get downloaded completely and is stuck in limbo. The video page shows "This video is being copied to the server storage and might not work properly" (translated that from german), the copy job shows as "completed" in the job list and there is no errors in the log (INFO level, see below).
The file situation after this is as follows, it seems like not all files did transfer:
On S3:
- video-segments-sha256.json
- video-master.m3u8
Locally:
- video-1080-fragmented.mp4
- video-1080.m3u8
Steps to reproduce
- Create Peertube with S3
- use create-move-video-storage-job
- See failed video
Describe the expected behavior
The video should download and be used locally
Additional information
-
PeerTube instance:
- URL: https://tube.bitwaves.de
- Version: 6.2.1
- NodeJS version: docker
- Ffmpeg version: docker
- S3 Provider: Strato HiDrive
-
Server Log:
[tube.bitwaves.de:443] 2024-09-06 08:58:04.902 info: Remote account https://mastodon.social/users/jaspershailesh1 removed.
[tube.bitwaves.de:443] 2024-09-06 08:59:48.787 info: Moving video 0a55c308-3f4a-4064-90b8-6a3c3f4e7a87 to file system in job 1.
[tube.bitwaves.de:443] 2024-09-06 08:59:48.901 info: Fetching HLS file hls/0a55c308-3f4a-4064-90b8-6a3c3f4e7a87/e14a8664-bdec-48a7-aa54-3e6387
d2af08-1080.m3u8 from object storage to /data/streaming-playlists/hls/0a55c308-3f4a-4064-90b8-6a3c3f4e7a87/e14a8664-bdec-48a7-aa54-3e6387d2af0
8-1080.m3u8.
[tube.bitwaves.de:443] 2024-09-06 08:59:50.913 info: Initialized S3 client https://s3.hidrive.strato.com with region us-east-1.
[tube.bitwaves.de:443] 2024-09-06 08:59:50.991 info: 2a02:26f7:ec60:6606:0:dad:1a33:c1f6 - - [06/Sep/2024:08:59:50 +0000] "GET / HTTP/1.0" 304
- "-" "Mozilla/5.0 (iPad; CPU OS 17_6_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.6 Mobile/15E148 Safari/604.1"
[tube.bitwaves.de:443] 2024-09-06 08:59:51.049 info: Fetching HLS file hls/0a55c308-3f4a-4064-90b8-6a3c3f4e7a87/e14a8664-bdec-48a7-aa54-3e6387
d2af08-1080-fragmented.mp4 from object storage to /data/streaming-playlists/hls/0a55c308-3f4a-4064-90b8-6a3c3f4e7a87/e14a8664-bdec-48a7-aa54-3
e6387d2af08-1080-fragmented.mp4.
Obviously, after getting this issue twice I don't have it anymore now that I posted here. The only thing I changed was renaming my videos folder to web-videos - the internal names of the web_videos variables were already changed, just the actual folders in s3 and on the server were still named videos. Could that have been it?
On some videos I get legit errors where the job state is "failed", the log looks like this:
ooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:440:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:629:24)",
"message": "The \"path\" argument must be of type string. Received null",
"code": "ERR_INVALID_ARG_TYPE"
}
}
[tube.bitwaves.de:443] 2024-09-06 10:42:05.467 error: Cannot execute job 12 in queue move-to-file-system. {
"payload": {
"videoUUID": "5f5c6c2b-84c2-47c6-ae7e-7e2845b28a50",
"isNewVideo": false,
"previousVideoState": 11
},
"err": {
"stack": "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string. Received null\n at new NodeError (node:internal/errors:405:5)\n at validateString (node:internal/validators:162:11)\n at join (node:path:1171:7)\n at VideoPathManager.getFSOriginalVideoFilePath (file:///app/dist/core/lib/video-path-manager.js:42:16)\n at moveVideoSourceFile (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:38:92)\n at moveToJob (file:///app/dist/core/lib/job-queue/handlers/shared/move-video.js:21:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Object.processMoveToFileSystem [as move-to-file-system] (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:15:5)\n at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:440:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:629:24)",
"message": "The \"path\" argument must be of type string. Received null",
"code": "ERR_INVALID_ARG_TYPE"
}
}
I don't know how to cancel this, I always get a warning about the failed download on that video page.
Yes, I got this from a video I just uploaded with the same peertube version:
[tube.bitwaves.de:443] 2024-09-13 08:15:39.507 error: Cannot move video https://tube.bitwaves.de/videos/watch/9813a2da-5a5a-468e-a1ab-ced22e2b325f storage. {
"err": {
"stack": "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string. Received null\n at new NodeError (node:internal/errors:405:5)\n at validateString (node:internal/validators:162:11)\n at join (node:path:1171:7)\n at VideoPathManager.getFSOriginalVideoFilePath (file:///app/dist/core/lib/video-path-manager.js:42:16)\n at moveVideoSourceFile (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:38:92)\n at moveToJob (file:///app/dist/core/lib/job-queue/handlers/shared/move-video.js:21:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Object.processMoveToFileSystem [as move-to-file-system] (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:15:5)\n at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:440:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:629:24)",
"message": "The \"path\" argument must be of type string. Received null",
"code": "ERR_INVALID_ARG_TYPE"
}
}
[tube.bitwaves.de:443] 2024-09-13 08:15:39.817 error: Cannot execute job 17 in queue move-to-file-system. {
"payload": {
"videoUUID": "9813a2da-5a5a-468e-a1ab-ced22e2b325f",
"isNewVideo": false,
"previousVideoState": 1
},
"err": {
"stack": "TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string. Received null\n at new NodeError (node:internal/errors:405:5)\n at validateString (node:internal/validators:162:11)\n at join (node:path:1171:7)\n at VideoPathManager.getFSOriginalVideoFilePath (file:///app/dist/core/lib/video-path-manager.js:42:16)\n at moveVideoSourceFile (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:38:92)\n at moveToJob (file:///app/dist/core/lib/job-queue/handlers/shared/move-video.js:21:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Object.processMoveToFileSystem [as move-to-file-system] (file:///app/dist/core/lib/job-queue/handlers/move-to-file-system.js:15:5)\n at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:440:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:629:24)",
"message": "The \"path\" argument must be of type string. Received null",
"code": "ERR_INVALID_ARG_TYPE"
}
}
However live stream recordings don't seem to have that issue, only video uploads. Automatic recordings of permanent live streams download fine. I do not have transcoding enabled for lives on the server.
And by the way to get rid of the "moving to storage failed" message on the video, that remains after the above error, I have to transcode to web video once and then delete it again. After that the message disappears.
Another by the way: It seems that even though I have web video disabled the server still creates web video files for imports. Later when I do a prune-storage a lot of web-video files are found that can be deleted.. Maybe thats the "path" it can't find?
Hi @normen, do you still have the issue?
Wow, I didn't check but I since updated to 7.0.0 and tried again now and all videos seem to download fine, thanks!