tubesync
tubesync copied to clipboard
meeb
Tasks are scheduled immediately but aren't running?
I added 7 sources a few days ago, for a total of 1320 videos. It seemed to be going along pretty well (apart from weirdly making/running tasks for thumbnails and metadata despite those boxes being unchecked in the source settings), but then randomly at 849 sources done, it's now stopped downloading media. I've restarted the container running TubeSync, I've restarted the VM running Docker running TubeSync, and I've restarted the physical server running everything, but no luck.
It says all the media download tasks are scheduled for running 'immediately'. There are no failed tasks and no active tasks. Looking at the logs of the docker container I don't see any errors. I don't really know how to get this going again?
Try resetting your task queue, there's a guide for it here:
https://github.com/meeb/tubesync/blob/main/docs/reset-tasks.md
Exact same issue here. The system queue all the task to be run immediately but nothing works. i tried to reset all the task queue with no joy. Problem is still there.
the log don’t show anything at all. For hours the container haven’t done practically nothing.
Can you check your process list to see if something like this is running?
app somepid 0.0 0.0 12345 67890 ? Ssl Day11 00:00 /usr/bin/python3 /app/manage.py process_tasks
That doesn't make a huge amount of sense. The process_tasks worker is managed by the process supervisor in the container which would restart it if it failed. Either way if it was dying it would leave significant evidence in the container logs. A reasonable next step would be to check the host system logs to see if something is killing it (OOM?) and to really check the container logs for errors. You can start the worker manually as a test in the container with something like:
$ docker exec -ti tubesync bash
- in container -
$ s6-setuidgid app python3 /app/manage.py process_tasks
If there's no work to do it'll sit in sleep() so when you control+c out of it after a while it'll show a trace that ends in
... spam ...
time.sleep(sleep)
KeyboardInterrupt
The only other thing I can think of is your clock is wrong I suppose, or the clock changed after the tasks were added. The tasks are processed from a database table based on a time / date stamp.
Either way, a reset_tasks on the command line will totally clear out and recreate any tasks if needed.
Before I’ve checked the host logs and not the container log. And there was nothing. The only thing that jump to my mind is that the server have a different time zone than the one setup in the container time zone environment setting.
Might try to change that… and then I’ll check all the logs and the command in the docker to see if I can find the way around it.
If I refresh the process list it does work, like I can see the list being cleared but new processes never start
What do you mean by "I can see the list being cleared"? If the list of tasks is going down then it's working.
I mean that I can see the list of task get deleted, the container then reschedule them as "immediate" but they just stay like that forever. Last time I've checked there was no change at all in more than 16 hours.
Seems that the app is start indexing the videos correctly, but after the index, nothing happen.
Restarting the container make it goes bit better but then it keep freezing.
This morning cleared all the tasks at around 8am and as of now, nothing happened since
Update: So checking now the task list, I can see that the system completed like more than 2000 tasks (all thumbnails) and still 309 are in the queue… but, the gui is not representing the thing correctly.
The container log is showing the thumbnail saved task, while in the interface the current running task is none. So either there is a delay form the gui to the actual application or there is something wrong. From 8am this morning till now, it’s seems quite strange that the system wasn’t able to download 21 videos.
The web UI is entirely static HTML. Unless you "catch" a page reload at the exact moment a thumbnail is downloading, which is quite a quick task, it will say "no task running" and is to be expected. At the moment you reload the page it is correct, the worker is probably sleeping for a second before the next thumbnail and technically no task is actively running. Media metadata and thumbnails are downloaded first, then media itself. If your tasks are being processed then it's working. The UI has no delay, the UI and the workers etc. all use the same (uncached) database. Wait for the metadata and thumbnail tasks to complete, then check the container logs again and see what the worker is doing. You should start to see either media being downloaded, or, in the web UI a reason why it's not being downloaded (download errors, no requested format available, etc.).
Tasks have to be processed quite slowly otherwise YouTube throttles or blocks your IP. TubeSync is primarily designed as a set-and-forget PVR style service.
I can confirm. Adjusting the timezone from Europe/London to Europe/Berlin have actually made everything working.
Thanks for your help!
Maybe what can be useful for everyone, leave the variable out of the docker compose, and let people add it if needed, because in my case, I haven't set it and it was already in Europe/London
Thanks again!
Interesting that fixed it for you. The date time comparisons should all be native even in the queue worker so that shouldn't, in theory, make any difference. I'll check the upstream code though just in case. It's part of the code I'm replacing soon anyway so hopefully, providing it doesn't affect anyone else, it'll be mitigated anyway.