Tdarr icon indicating copy to clipboard operation
Tdarr copied to clipboard

Transcode cache not always cleaned up

Open Glynax opened this issue 3 years ago • 4 comments
trafficstars

Describe the bug A clear and concise description of what the bug is. I have ~20 nodes all using a ZFS mirrored 1TB SSDs NFS share for a temp transcode cache, I notice every few weeks or months it inevitably fills up (till I catch it at 80%) with files that have not been touched in weeks. Picture attached shows a handful of those small null files but the bulk of the space is taken by videos at various levels of processing.

To Reproduce Steps to reproduce the behavior: I'm just running tdarr and nodes in a docker compose (except my windows node)

Expected behavior A clear and concise description of what you expected to happen. I'd expect the temp files to be purged after the entire plugin stack is finished for said file

Screenshots If applicable, add screenshots to help explain your problem. https://cdn.discordapp.com/attachments/623799920574595092/972665664529645578/unknown.png

Tdarr_Server_Log1.txt Tdarr_Server_Log2.txt Tdarr_Server_Log.txt

Please provide the following information:

  • Config files [can be found in /app/configs/ when using Docker or in the /configs folder next to Tdarr_Updater if not using Docker] image

  • Log files [can be found in /app/logs/ when using Docker or in the /logs folder next to Tdarr_Updater if not using Docker]

-Worker error [can be found on the 'Tdarr' tab by pressing the 'i' button on a failed item in the staged file section or in the transcode error section at the bottom] I had just cleared out the staging queue.. Is there anywhere else I could find this info?

  • OS: [e.g. Linux] Ubuntu
  • Browser [e.g. chrome, safari] Chrome
  • Version [e.g. 2.00.00] 2.00.18

Additional context Add any other context about the problem here.

Glynax avatar May 08 '22 05:05 Glynax

Those files all have null as the container so are they from when you didn't have the plugin configs quite right? If you clear those out manually do you still encounter the issue on new files?

HaveAGitGat avatar May 16 '22 03:05 HaveAGitGat

Those files all have null as the container so are they from when you didn't have the plugin configs quite right? If you clear those out manually do you still encounter the issue on new files?

I still have it to this day, haven't changed anything in awhile I was letting it settle to see if it would go away. image

Glynax avatar Jun 03 '22 04:06 Glynax

I have this exact same issue. This is over three days:

image

ArmonicPvP avatar Jul 09 '22 00:07 ArmonicPvP

Can confirm that I also see extra files left behind occasionally. It usually cleans up, but sometimes will leave a file behind. I haven't found a specific cause yet though.

cjmanca avatar Sep 27 '22 01:09 cjmanca

For me this happens for most files. I set my shared memory as the transcode location, so this becomes problematic in under 1 hour of transcoding files, sometimes within minutes. This happens even if I only allow for 1 transcode and 0 for everything else.

janmartenjongerius avatar Dec 02 '22 16:12 janmartenjongerius

2.00.20 has several new things to address this:

- Added improved cache cleaner service
- Added cache file browser to Staging section for manually deleting cache files if needed

Cache cleaner service will run hourly and clean up any dangling cache files not in use. Then there's a UI in the staging section to force delete cache files if needed.

HaveAGitGat avatar Mar 13 '23 20:03 HaveAGitGat