MUMC icon indicating copy to clipboard operation
MUMC copied to clipboard

Not deleting unplayed media past creation data criteria

Open Doug411 opened this issue 1 year ago • 119 comments

Hi again! I have a folder that I am populating with podcasts. I'm using TubeArchivist to download them to a Podcast Library. Each individual podcast is a series folder on Jellyfin. Many of the podcasts are daily. I'd like to set a retention period for 14 days, and delete old content regardless of whether it has been watched or not (unless its favorited), I've tried various incarnations, but the only items it seems to put in my delete queue are items that were watched by the monitored user.

This is what I thought would work based on my understanding of the logic. (turn off played evaluation, set my created condition, and put the count <0). However, it only flagged the played items.

episode: played: condition_days: -1 count_equality: '>=' count: 1 created: condition_days: 14 count_equality: '>=' count: -1 behavioral_control: true

Also, what is the best way to potentially change the retention period at the podcast level? eg

Podcast (Jellyfin Series Library) -Podcast 1 (Sub Folder) 14 days -Podcast 2 (Sub Folder) 21 days -Podcast 3 (Sub Folder) 30 days

I also want to clear up my understanding of how the script works. If I use tags, do I still have to blacklist the library that I'm evaluating? In other words is it possible to NOT blacklist the podcast library, but blacktag items in a library that is whitelisted?

What happens if I tag the individual podcast folders, versus tagging the items? Tubeachivist doesn't have functionality to tag items upon download. I'm trying to figure out the best way to accomplish a folder based retain/purge logic. ideas?

I also want to understand more why you are allowing multiple monitor users. If I monitor 5 users played status.... Is the main use case to not delete until all 5 have watched and it meets the days criteria? Wow, that's complex if so.

Doug411 avatar Jun 03 '24 04:06 Doug411

Hey there!

Attach or send a pastebin link to your config file (please do not copy/paste). Then I will be able to see exactly what settings are there/not there and can help you get MUMC working the way you want it to.

I will be able to answer your other questions after work today.

terrelsa13 avatar Jun 03 '24 11:06 terrelsa13

Thanks for the help. Much appreciated! see patebin link. I had tags enabled at one point, and got an error, so I just blacklisted the one folder. I was trying to get it working in its simplest form, before I ventured too far. I still would like clarity on tags and blacklists (do I need to blacklist if I blacktag at the item level? Can I black tag a folder, etc). Also unclear on how it works with multiple users. I read through the help but wasn't quite sure of the end result. (https://pastebin.com/SB7bqyy5)

Doug411 avatar Jun 03 '24 20:06 Doug411

FYI - I do not see a pastebin link in your response.

Answers to your questions:

I also want to clear up my understanding of how the script works. If I use tags, do I still have to blacklist the library that I'm evaluating? In other words is it possible to NOT blacklist the podcast library, but blacktag items in a library that is whitelisted?

  • It depends on how your configuration is setup.
  • media_items in whitelisted libraries can be searched for blacktags and/or whitetags or neither.
  • media_items in blacklisted libraries can be searched for blacktags and/or whitetags or neither.

What happens if I tag the individual podcast folders, versus tagging the items? Tubeachivist doesn't have functionality to tag items upon download. I'm trying to figure out the best way to accomplish a folder based retain/purge logic. ideas?

  • Tagging a parent applies to all of its children.
    • I see I don't explain this in the wiki; but the parent-child relationship applies to tags and favorites.
  • Take a look at Blacktagging vs Whitetagging.

I also want to understand more why you are allowing multiple monitor users. If I monitor 5 users played status.... Is the main use case to not delete until all 5 have watched and it meets the days criteria? Wow, that's complex if so.

  • Short answer; yes. MUMC allows several combinations of options to specify when a media_item should be deleted.

    • Everyone wants to manage their media differently.
  • TLDR; Originally this script started out monitoring a single user; then the requests came to make it work for multiple users.

    • More or less; people want a way to account for users who consume media_items at different speeds.
    • There are some limitations that have to be handled manually; either using favorites or tags.
      • For instance, without favorites or tags there is no good way to determine when to delete an episode if user1 has watched it but user2 either does not have time to watch it before the script runs again. Or just as likely user2 never intends on watching it.
        • To make something like this work the script would have to walk thru assigning individual series to each user.
        • And anytime a user or series is added this would need to be repeated.
        • The below explains how MUMC avoids going down this path.
  • To keep things relatively simple there are behavioral_statements.

  • At a high level these allow a flexible way to do things like this:

Example 1

advance_settings:
 behavioral_statements:
   episode:
     conditional_behavior: favorite
      action: delete
      user_conditional: any
      played_conditional: all_any
      action_control: 3
      dynamic_behavior: false
  • user_conditional
    • If any monitored_user has the media_item favorited and...
  • played_conditional
    • All monitored_users meet either the played filter statement or created filter statement
  • Then follow the desired action_control
    • 3 - Action taken on True; No action taken on False
  • action
    • In this case the action is to delete when True (aka add the media_item to the delete list)
    • In this case the action is to take no action on False (nothing is done to the media_item; it will remain in or out of the delete list)

Example 2

advance_settings:
 behavioral_statements:
   episode:
     conditional_behavior: favorite
      action: delete
      user_conditional: any
      played_conditional: all_all
      action_control: 5
      dynamic_behavior: true
  • user_conditional
    • If all monitored_users have the media_item favorited and...
  • played_conditional + (dynamic_behavior)
    • All monitored_users + (who have this media_item favorited) meet both the played filter statement and created filter statement
  • Then follow the desired action_control
    • 5 - Action taken on True; Opposite action taken on False
  • action
    • In this case the action is to delete when True (aka add the media_item to the delete list)
    • In this case the action is to keep when False (aka remove the media_item from the delete list)

Example 3

advance_settings:
 behavioral_statements:
   episode:
     conditional_behavior: whitelist
      action: keep
      user_conditional: any
      played_conditional: all_played
      action_control: 5
      dynamic_behavior: true
  • user_conditional
    • If any monitored_user has the media_item whitelisted and...
  • played_conditional + (dynamic_behavior)
    • All monitored_users + (who have the library this media_item is in whitelisted) met the played filter statement
  • Then follow the desired action_control
    • 5 - Action taken on True; Opposite action taken on False
  • action
    • In this case the action is to keep when True (aka remove the media_item from the delete list)
    • In this case the action is to delete when False (aka add the media_item to the delete list)

Also, what is the best way to potentially change the retention period at the podcast level? eg Podcast (Jellyfin Series Library) -Podcast 1 (Sub Folder) 14 days -Podcast 2 (Sub Folder) 21 days -Podcast 3 (Sub Folder) 30 days

There are two options that come to mind.

Option#1

  • Set up x3 instances of MUMC (e.g. MUMC14, MUMC21, MUCM30)

  • Create three blacktags (e.g. "14 Day Delete", "21 Day Delete", "30 Day Delete")

    • Tag the Podcast 1 library with the "14 Day Delete" tag
    • Tag the Podcast 2 library with the "21 Day Delete" tag
    • Tag the Podcast 3 library with the "30 Day Delete" tag
  • In the MUMC14 instance setup the config as shown below:

basic_settings:
  filter_statements:
    episode:
      created:
        condition_days: 14
        count_equality: '>='
        count: 0
        behavioral_control: true
advanced_settings:
  filter_statements:
    episode:
      query_filter:
        favorited: true
        whitetagged: false
        blacktagged: true
        whitelisted: false
        blacklisted: false
  behavioral_statements:
    episode:
      favorited:
        action: keep
        user_conditional: any
        played_conditional: ignore
        action_control: 3
      blacktagged:
        action: delete
        user_conditional: any
        played_conditional: any_created
        action_control: 5
        dynamic_behavior: false
        tags: ["14 Day Delete"]
  delete_empty_folders:
    episode:
      season: true
      series: true
  • Repeat for the MUMC21 and MUMC30 instances.
    • Make sure to change the created condition days and the episode tag for each.

Option#2

  • Set up x3 instances of MUMC (e.g. MUMC14, MUMC21, MUCM30)

  • Create x3 libraries

    • Podcast14 Library; put podcasts intended to be deleted after 14 days here
    • Podcast21 Library; put podcasts intended to be deleted after 21 days here
    • Podcast30 Library; put podcasts intended to be deleted after 30 days here
  • During setup of the MUMC14 instance only blacklist the "Podcast14 Library"

  • In the MUMC14 instance setup the config as shown below:

basic_settings:
  filter_statements:
    episode:
      created:
        condition_days: 14
        count_equality: '>='
        count: 0
        behavioral_control: true
advanced_settings:
  filter_statements:
    episode:
      query_filter:
        favorited: true
        whitetagged: false
        blacktagged: false
        whitelisted: false
        blacklisted: true
  behavioral_statements:
    episode:
      favorited:
        action: keep
        user_conditional: any
        played_conditional: ignore
        action_control: 3
      blacklisted:
        action: delete
        user_conditional: any
        played_conditional: any_created
        action_control: 5
        dynamic_behavior: false
  delete_empty_folders:
    episode:
      season: true
      series: true
  • Repeat for the MUMC21 and MUMC30 instances.
    • Make sure to change the created condition days for each.

terrelsa13 avatar Jun 03 '24 22:06 terrelsa13

To be clear, you only need to run x3 separate instances of MUMC if they are docker containers.

When running outside of a docker container, multiple config files can be created. And then use the -c command line option. *mumc_config14.yaml *mumc_config21.yaml *mumc_config30.yaml

MUMC has Command Line Options which allow specifying a config file to pass into it.

/path/to/python /path/to/mumc.py -c /path/to/mumc_config14.yaml
/path/to/python /path/to/mumc.py -c /path/to/mumc_config21.yaml
/path/to/python /path/to/mumc.py -c /path/to/mumc_config30.yaml

terrelsa13 avatar Jun 03 '24 22:06 terrelsa13

Thanks. Very Helpful. Can you see my pastebin? It was in pasted into a codeblock before I think. I took it out and it appears as a link now.

In theory I understand everything you described. However, I still don't understand why my setup was only flagging watched content for deletion.

To simplify my setup to start, this is what I want to have happen. I would like to monitor 5 users for (favoriting action only). On the Youtube Library, I'd like to add 14 day delete tags to SOME of the subfolders (Podcast 1,2,3). Let's say I have also (Podcast 4,5,6), that for now I would add no tag to (or perhaps I need to add a whitetag?).

I want the content in folders 1,2,3 to be purged after 14 days unless it is in someone's favorites or the individual item is white tagged. I don't want to consider watched status.

If I can get some help on that (with my pastebin config), I should have no trouble following the example above to extend it for different retention periods if I desire. It looks like I'll end up going with your option 1, as the tubearchivist solution is pretty heavy (lots of ram for indexing and elastic search). It creates subfolders under 1 parent. Running multiple instances of tubearchivist to create multiple libraries would be a big load.....and alternatively I don't want each podcast to be its own individual library (i'd want them grouped logically in jellyfin). So I think the tagging example would work well (as long as I can just tag the podcast folder (below the library and above the content items as in the example below.

Video Podcast (Jellyfin library folder named 'Youtube' on Server)
   -Podcast 1 (BLACKTAG)
      -video 1
      -video 2 (Favorite)
      -video 3 (whitetag)

   -Podcast 2 (BlACKTAG)
      -video 1
      -video 2

   -Podcast 3 (BLACKTAG)
      -video 1
      -video 2 

  -Podcast 4 (no tag?)
      -video 1
      -video 2

  -Podcast 5 (no tag?)
      -video 1
      -video 2

  -Podcast 6 (no tag?)

Doug411 avatar Jun 03 '24 23:06 Doug411

Yep! I can see the pastebin now. Much appreciated. I will re-read this tomorrow evening after work to see how I can help you.

terrelsa13 avatar Jun 04 '24 00:06 terrelsa13

In general your request is possible. Just not exactly how you have described using Jellyfin.

Jellyfin does not allow sub-library-folder actions via the GUI (see this related Known Limitation). If you want to favorite or tag sub-library-folders you will either need to switch to Emby or write your own script to manipulate the API to tag the sub-library-folders (which I'm not 100% sure Jellyfin would let happen via the API either).

With Jellyfin tv shows and movies you can tag:

  • Top Level Library Folders
  • Series Folders
  • Season Folders
  • Episodes
  • Movies

To simplify my setup to start, this is what I want to have happen. I would like to monitor 5 users for (favoriting action only). On the Youtube Library, I'd like to add 14 day delete tags to SOME of the subfolders (Podcast 1,2,3). Let's say I have also (Podcast 4,5,6), that for now I would add no tag to (or perhaps I need to add a whitetag?).

I want the content in folders 1,2,3 to be purged after 14 days unless it is in someone's favorites or the individual item is white tagged. I don't want to consider watched status.

  • Assuming you will be tagging/favorting series folders instead of sub-library-folders.
  • Assuming the Youtube library is whitelisted for all x5 users. This also means all sub-library-folders are whitelisted (again Known Limitation with Jellyfin sub-library-folders).
  • Assuming the Youtube library is considered a tv library with "series", "seasons", and "episodes" (you mentioned each podcast is in its own series folder; what about season folder?).

I removed the admin_settings portion from my example config. The config is yaml, it is meant to be human readable and easy to understand. You should be able to copy/paste my example below into your config. It is ok to completely overwrite the basic_settings and advanced_settings sections. Just don't overwrite the admin_settings, version, or DEBUG section sections.

mumc_config.yaml.txt

terrelsa13 avatar Jun 04 '24 03:06 terrelsa13

I looked at your config and I understand it. I was missing logic in the behavioral statements. Developing a script that will work across both Jellyfin and Emby with the flexibility to handle multiple users and scenarios, is no easy task. Kudos. I'm iimpressed.

I'm not sure I fully understand the logic of action control 3 vs 5 for keep vs delete.... I'll have to play with it some. FYI.. I tried the docker and got an error after choosing people/libraries. I'll try the native version and let you know if it finishes.



Traceback (most recent call last): File "/usr/src/app/mumc_modules/mumc_config_import.py", line 93, in importConfig import mumc_config as cfg ModuleNotFoundError: No module named 'mumc_config'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/src/app/./mumc.py", line 147, in MUMC() File "/usr/src/app/./mumc.py", line 29, in MUMC cfg,init_dict=importConfig(init_dict,cmdopt_dict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/mumc_modules/mumc_config_import.py", line 109, in importConfig importHasException(init_dict,cmdopt_dict) File "/usr/src/app/mumc_modules/mumc_config_import.py", line 24, in importHasException build_configuration_file(init_dict) File "/usr/src/app/mumc_modules/mumc_config_builder.py", line 153, in build_configuration_file yaml_configurationBuilder(the_dict) File "/usr/src/app/mumc_modules/mumc_configuration_yaml.py", line 380, in yaml_configurationBuilder save_yaml_config(config_data,str(the_dict['mumc_path']) + '/' + the_dict['config_file_name_yaml']) File "/usr/src/app/mumc_modules/mumc_paths.py", line 119, in save_yaml_config with open(filePathName,'w') as file: ^^^^^^^^^^^^^^^^^^^^^^ IsADirectoryError: [Errno 21] Is a directory: '/usr/src/app/mumc_config.yaml'

Doug411 avatar Jun 04 '24 21:06 Doug411

nevermind on the docker issue. I didn't have the empty file out there. working through some yaml issues....

Doug411 avatar Jun 04 '24 22:06 Doug411

Is all of this required? I remember reading something about having changed to minimal config. I guess I can run through the setup again, and be even more surgical. I remember questionining whether I should leave in the filtering/favorites advanced pieces.

ConfigNameError: basic_settings > filter_statements > episode > played > condition_days is missing from mumc_config.yaml ConfigNameError: basic_settings > filter_statements > episode > played > count_equality is missing from mumc_config.yaml ConfigNameError: basic_settings > filter_statements > episode > played > count is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > season_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > series_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > extra > library_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > studio_network is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > studio_network_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitetagged > tags is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacktagged > tags is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > action_days is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > user_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > played_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > action_days is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > user_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > played_conditional is missing from mumc_config.yaml

Doug411 avatar Jun 04 '24 23:06 Doug411

What version are you using? The minimum config feature does not work until >= v5.8.0-beta. If using Docker check out this post from the last issue:

terrelsa13 avatar Jun 05 '24 00:06 terrelsa13

Very strange. I was running it native when it generated those errors. Then I ran it in docker and it didn't generate any yaml in the advanced section at all (not even for the blacktags and white tags for which it prompted in the script. I pasted in your code and it ran first go. Its processing a lot of episodes. I think it runs for a bit and restarts. I tried going into the shell to try to find a log. However, it restarts before I can locate one. What's the best way to determine what's flagged for deletion with the docker version? One thought would be instead of mounting to the config.yaml, mount to a config folder. The script could look for the mmu_config.yaml in the config folder, and output the log to the same folder. The back to back restarts seem a bit problematic for diagnosis though. Not sure if it wipes the logs on each run or adds to them (as I haven't been able to get to a log). Thoughts?

Doug411 avatar Jun 05 '24 01:06 Doug411

fyi, tried watching the log in portainer, but its moving through episodes too fast for me to analyze, and then restarts

Doug411 avatar Jun 05 '24 01:06 Doug411

Very strange. I was running it native when it generated those errors.

Native version >= 5.8.0-beta? Or <= 5.7.3?

Then I ran it in docker and it didn't generate any yaml in the advanced section at all (not even for the blacktags and white tags for which it prompted in the script.

Ahhh good catch here. Script is not saving the config sections which are not longer set to default values during script creation.

I tried going into the shell to try to find a log.

The mumc_DEBUG.log should be in the same location as the mumc_config.yaml. Create the mumc_DEBUG.log file outside of docker. In the same location as mumc_config.yaml. Then add a new volume:

volumes:
   - "/opt/MUMC/mumc_config.yaml:/usr/src/app/mumc_config.yaml"
   - "/opt/MUMC/mumc_DEBUG.log:/usr/src/app/mumc_DEBUG.log"

The back to back restarts seem a bit problematic for diagnosis though. Not sure if it wipes the logs on each run or adds to them (as I haven't been able to get to a log). Thoughts?

Yeah, it definitely deletes any existing mumc_DEBUG.log anytime MUMC runs. So my above suggestion will make the debug file accessible outside of the container. But it will be difficult to see the deletion summary before MUMC runs again.

One thought would be instead of mounting to the config.yaml, mount to a config folder. The script could look for the mmu_config.yaml in the config folder, and output the log to the same folder.

I considered this. But didn't have the time/patients to mess with the configuration import logic at the time.

Looks like there are still a few things needing to be worked out with the containerized script. I likely will not be able to make any good headway on this until the weekend.

terrelsa13 avatar Jun 05 '24 02:06 terrelsa13

fyi, tried watching the log in portainer, but its moving through episodes too fast for me to analyze, and then restarts

I thought the portainer logs for a container could be paused? Even if it moves thru episodes quickly the portainer logging page has a buffer which allows scrolling back some number of lines.

terrelsa13 avatar Jun 05 '24 02:06 terrelsa13

If the portainer log buffer doesn't go back far enough, try viewing the container logs in a terminal window.

Live View

docker logs -f mumc

Staic View Of The Last 10 Minutes

docker logs --since 10m mumc

Save Static View Of The Last 10 Minutes

docker logs --since 10m mumc > /path/to/save_file.log

terrelsa13 avatar Jun 05 '24 02:06 terrelsa13

i changed the restart policy to no and exported the logs. It processes through files for 4 of 6 users. It gets an error on user 5, stops and restarts again. The error is below. Even more dissappointing is that the filter isnt picking up any of the blacktags on the series folder. Answering your question from ealier today, the structure is Just Library, Series (No Season). I'm running the tubearchivist plugin for jellyfin. It is creating Season folders in the Jellyfin UI (but its not stored that way on disk). All episodes for a series (across all seasons) are in one Series folder along with associated metadata and subtitles. I added the blacktags to the series folder using the meta data manager in jellyfin. I could have sworn I saw some data flying by earlier today in portainer that looked like some blacktags were being assigned. But its definately not doing it now (i've run it many times, and have the cumulative logs)

2024-06-05T01:35:53.010147080Z Mitchell xxxx-xxxx - ca7ad35ea9xxxxxxxxxxxxxxxxxxxx 2024-06-05T01:35:53.010149250Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010151509Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010153739Z 2024-06-05T01:35:53.010155830Z HTTPError: Unable to get information from server during processing of: season_info_for_32d219223d3a18d2102c7a1f6d444a1e 2024-06-05T01:35:53.010158557Z Object: <bound method Request.header_items of <urllib.request.Request object at 0x7d66163b5ad0>> 2024-06-05T01:35:53.010161297Z URL: http://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type 2024-06-05T01:35:53.010165960Z Method: 2024-06-05T01:35:53.010168346Z Header: {'Authorization': 'MediaBrowser Client="mumc.py", Device="Multi-User Media Cleaner", DeviceId="MUMC", Version="5.8.9-beta", Token="abcf86fab5a44978b95f502caf140e6b"', 'Content-type': 'application/json'} 2024-06-05T01:35:53.010171114Z Data: None 2024-06-05T01:35:53.010173308Z 2024-06-05T01:35:53.010175420Z HTTPError: 404 - Not Found 2024-06-05T01:35:53.010177684Z 2024-06-05T01:35:53.010179822Z Check password and/or remove any GUI API keys for MUMC 2024-06-05T01:50:32.616000519Z 2024-06-05T01:50:32.616025448Z ----------------------------------------------------------------------------------------

Doug411 avatar Jun 05 '24 02:06 Doug411

Also, are tags User based, or if one user creates a tag is it there for all users? Just curious..... I did search on Blacktags across all the cumulative rows. I definately don't have any yes'. I was more curious of what to expect to see when its working.

Doug411 avatar Jun 05 '24 02:06 Doug411

FYI. I was unable to mount the debug log the way indicated above. I tried creating the file, but I get an error message indicating device, resource busy. If the file isn't there, it creates a folder upon execution.

I kind of think the optimal way to run the docker solution would be to set the restart policy to no (as I have it). A restart can just be scheduled with cron (as you do the native solution). This would be perfect if the log file was viewable in the mounted volume. It would be just like native (without having to worry about dependencies)...

Now if I can just get it to recognize my blacktags.... :) help... :)

FYI... I may run it on my normal TV Shows as well. I need to look if that library has season folders. Is it going to be a problem if one library does and one doesn't have season folders?

Doug411 avatar Jun 05 '24 03:06 Doug411

FYI. This is how the podcasts are organized in the GUI. I attempted to use two different tags as depicted (auto-purge and Retain14d). When MUMC runs it doesn't assign either tag to the underlying items. All episodes are marked as not being blacktagged. The tags are on the series item.

PodcastJellyfin

Doug411 avatar Jun 05 '24 15:06 Doug411

i changed the restart policy to no and exported the logs. It processes through files for 4 of 6 users. It gets an error on user 5, stops and restarts again. The error is below. Even more dissappointing is that the filter isnt picking up any of the blacktags on the series folder. Answering your question from ealier today, the structure is Just Library, Series (No Season). I'm running the tubearchivist plugin for jellyfin. It is creating Season folders in the Jellyfin UI (but its not stored that way on disk). All episodes for a series (across all seasons) are in one Series folder along with associated metadata and subtitles. I added the blacktags to the series folder using the meta data manager in jellyfin. I could have sworn I saw some data flying by earlier today in portainer that looked like some blacktags were being assigned. But its definately not doing it now (i've run it many times, and have the cumulative logs)

2024-06-05T01:35:53.010147080Z Mitchell xxxx-xxxx - ca7ad35ea9xxxxxxxxxxxxxxxxxxxx 2024-06-05T01:35:53.010149250Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010151509Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010153739Z 2024-06-05T01:35:53.010155830Z HTTPError: Unable to get information from server during processing of: season_info_for_32d219223d3a18d2102c7a1f6d444a1e 2024-06-05T01:35:53.010158557Z Object: <bound method Request.header_items of <urllib.request.Request object at 0x7d66163b5ad0>> 2024-06-05T01:35:53.010161297Z URL: http://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type 2024-06-05T01:35:53.010165960Z Method: 2024-06-05T01:35:53.010168346Z Header: {'Authorization': 'MediaBrowser Client="mumc.py", Device="Multi-User Media Cleaner", DeviceId="MUMC", Version="5.8.9-beta", Token="abcf86fab5a44978b95f502caf140e6b"', 'Content-type': 'application/json'} 2024-06-05T01:35:53.010171114Z Data: None 2024-06-05T01:35:53.010173308Z 2024-06-05T01:35:53.010175420Z HTTPError: 404 - Not Found 2024-06-05T01:35:53.010177684Z 2024-06-05T01:35:53.010179822Z Check password and/or remove any GUI API keys for MUMC 2024-06-05T01:50:32.616000519Z 2024-06-05T01:50:32.616025448Z ----------------------------------------------------------------------------------------

Ok, I will need your help getting a mumc_DEBUG.log during a run that is generating this error.

  1. In the config set DEBUG: 4
  2. Run the script
  3. Wait for the error to happen
  4. Either attach the mumc_DEBUG.log to your reply OR pastebin it and reply with the link. (no direct copy/paste into the reply textbox please, it always messes up the formatting and makes it difficult to decipher what is happening.)

I will also need a copy the mumc_config.yaml. Same thing, either attach to your reply or pastebin.

Also, are tags User based, or if one user creates a tag is it there for all users? Just curious..... I did search on Blacktags across all the cumulative rows. I definately don't have any yes'. I was more curious of what to expect to see when its working.

  • Tags apply to the media_item, series, season, etc... So they are global, not user specific.
  • Tagging

I kind of think the optimal way to run the docker solution would be to set the restart policy to no (as I have it). A restart can just be scheduled with cron (as you do the native solution). This would be perfect if the log file was viewable in the mounted volume. It would be just like native (without having to worry about dependencies)...

I agree with your suggestion. Truthfully did not know this was possible. Thanks for the feedback!

I need to look if that library has season folders. Is it going to be a problem if one library does and one doesn't have season folders?

I took a look at the tag logic. It should not matter if there is a season folder.

When a media_item is returned during the query:

  1. MUMC checks media_item for tags that match what's in the config
  2. Then (assuming episode) looks for the season attribute.
    1. If the season attribute is not there it looks for the generic parent attribute.
  3. MUMC then requests the data for the season/parent
  4. Then checks if the season/parent has tags that match what's in the config
  5. The season/parent is then check for its 'seriesorparentattribute and this repeats until we get to the top (aka thelibrary`)

All episodes are marked as not being blacktagged. The tags are on the series item.

The debug logs I requested at the top of this reply will help me understand why MUMC is not considering the media_items tagged. But also make sure there is no rouge spaces, tabs, underscores, etc... Also if there are quotes around the tags in the config, try removing the quotes and see if that helps.

terrelsa13 avatar Jun 05 '24 20:06 terrelsa13

Thanks so much for all your help. I don't know if the portainer logs are the same as the debug.log. I cant get to the debug.log file with the docker version, as it didn't run when I tried mounting it to an empty file. I'll try the native version. Looking back at your prior comment, seems I was running the 5.7.3 version. Should I clone a different version? 5.8.0-beta?

Doug411 avatar Jun 06 '24 03:06 Doug411

config https://pastebin.com/iGYVQYC2

the debug log is 354MB, too big..... mostly because it is considering my regular shows folder. Is there a way to get it to run on a subset? It's too big for my chromebook to manipulate.

I shared the log to your gmail account listed on this site. You should get a link to the doc in drive. Hopefully you have a little more powerful pc to navigate it.

Doug411 avatar Jun 06 '24 06:06 Doug411

Thanks so much for all your help.

Np! Been in your shoes.

I don't know if the portainer logs are the same as the debug.log.

Nope. They are different.

I'll try the native version. Looking back at your prior comment, seems I was running the 5.7.3 version. Should I clone a different version? 5.8.0-beta?

Yes, for now let's stick to the native version MUMC v5.8.9. This way we are not fighting Docker "unknowns" and can figure out the no-tags issues.

the debug log is 354MB, too big.....

My b. I should have mentioned it may become a big file.

mostly because it is considering my regular shows folder. Is there a way to get it to run on a subset? It's too big for my chromebook to manipulate.

Yeah we can disable all libraries for all users. Then enable the "youtube"/podcast library only for 1-2 users to keep it simple.

  1. Look for lib_enabled: true in the config file.
  2. Do a find/replace
    1. Find lib_enabled: true
    2. Replace with lib_enabled: false
  3. Pick 1-2 users
  4. Find the "youtube"/podcast library for them
  5. Set the lib_enabled: true just for the "youtube"/podcast library of the 1-2 users you've chosen
  6. Now re-run with DEBUG: 4
  7. The mumc_DEBUG.log should be a more manageable size

As an example. This is what the "youtube"/podcast library entry in the config looks like. lib_enabled: true is the last variable for the library:

    - lib_id: 608e1ed1e97ee5c4556f3d0299ac9894
      collection_type: tvshows
      path: /media/data/media/youtube
      network_path: null
      lib_enabled: true

You want to set lib_enabled: false for all libraries. Then set lib_enabled: true only for the "youtube"/podcast library for 1-2 users.

I shared the log to your gmail account listed on this site. You should get a link to the doc in drive. Hopefully you have a little more powerful pc to navigate it.

Currently trying to open it. It's been spinning for 15mins. I'll let it keep trying for a little while longer. Probably better to try to do the steps I pointed out above; which should make a much smaller mumc_DEBUG.log file.

terrelsa13 avatar Jun 06 '24 21:06 terrelsa13

will do. FYI, the big file opens when downloaded local. Hard to use 'find' function in it though. I'm using a chromebook with only 8gb ram, and it was really slow on finds (but it opened in about a minute or so,

Doug411 avatar Jun 06 '24 22:06 Doug411

FYI, I sent another version with the suffix admin to your drive/Gmail. It's 50mb, so much smaller. It doesn't have the hard error in it, but should be useful in diagnosing the BLACKTAG issue. I'm going to try to generate the hard error now, with the user that errored out in the original post.

Doug411 avatar Jun 06 '24 23:06 Doug411

sent another one with the hard error in it.

Doug411 avatar Jun 06 '24 23:06 Doug411

Ok. Making some progress. I was able to figure out the tagging issue. I have a local v5.8.10-beta I have not pushed to Github yet, which has the fix for the missed tagging. Hoping to figure out this hard error before pushing it.

Now looking at the hard error. It appears to be happening when the script is it trying to get the "season04" folder information for the episode "Seinfeld - S04E01" for the Mitchell user.

Script is generating a HTTPError: 404 - Not Found error. Here is what we are going to try:

  • First we will recreate the URL using the Admin user_id
  • Then recreate the URL with the Mitchell user_id

Instructions:

  1. Open a browser on a machine that can access your Jellyfin server
  2. Copy/paste this URL to simulate the Admin user asking for the "season04" info:
    1. If any of the ids were modified, be sure the un-modify them in the URLs below
http://localhost:8096/Users/74d15d7a864441ccbabfc7122b9136af/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type&api_key=YOUR_API_KEY_HERE
  1. Open a new browser tab.
  2. Copy/paste this URL to simulate the Mitchell user asking for the "season04" info:
http://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type&api_key=YOUR_API_KEY_HERE
  1. If using chrome browser, hit the "Pretty-print" checkbox in the top left corner to make it easier to read
  2. Does Jellyfin return the Seinfeld Season04 data for the Admin user?
  3. Does Jellyfin return the Seinfeld Season04 data for the Mitchell user?
  4. Does the data match for both users?

terrelsa13 avatar Jun 07 '24 02:06 terrelsa13

Returns for admin. Does not return for the Mitch user. I do have parental controls turned on for Mitch account. I will try turning it off and trying again......

{ "type": "https://tools.ietf.org/html/rfc9110#section-15.5.5", "title": "Not Found", "status": 404, "traceId": "00-1430f6134780f1c331c19f10d6194781-17a075021270fb37-00" }

OK, just redid it with parental controls turned off. It returns the same. The issue is with parental controls.

Doug411 avatar Jun 07 '24 03:06 Doug411

was the tagging issue hard to fix?

Doug411 avatar Jun 07 '24 03:06 Doug411