pinchflat
pinchflat copied to clipboard
Published
20 hours ago •
kieraneglin
kieraneglin
[Triage] Indexing Broken?
Describe the bug Adding new sources no longer indexes them and adds them to the download queue
To Reproduce Add a source Indexing repeats multiple times until limits / timeouts are reached. Nothing is downloaded and nothing is added to the queue.
Based on logs it seems to be finding the videos but no downloads start.
Expected behavior Source is indexed and downloads start
Diagnostic info
- App Version: 2025.3.17
- yt-dlp Version: 2025.03.31
- Apprise Version: v1.9.2
- System Architecture: x86_64-pc-linux-gnu
- Timezone: America/New_York
Additional context
This just repeats over and and over in the logs:
17:40:10.784 [debug] Replied in 1ms
17:40:11.505 [debug] EOF reached, waiting before trying to read new lines
17:40:12.506 [debug] EOF reached, waiting before trying to read new lines
17:40:13.507 [debug] EOF reached, waiting before trying to read new lines
17:40:14.508 [debug] EOF reached, waiting before trying to read new lines
17:40:15.509 [debug] EOF reached, waiting before trying to read new lines
17:40:16.510 [debug] EOF reached, waiting before trying to read new lines
17:40:17.511 [debug] EOF reached, waiting before trying to read new lines
17:40:18.512 [debug] EOF reached, waiting before trying to read new lines
17:40:19.513 [debug] EOF reached, waiting before trying to read new lines
17:40:20.514 [debug] EOF reached, waiting before trying to read new lines
17:40:21.515 [debug] EOF reached, waiting before trying to read new lines
17:40:22.516 [debug] EOF reached, waiting before trying to read new lines
17:40:23.517 [debug] EOF reached, waiting before trying to read new lines
17:40:24.518 [debug] EOF reached, waiting before trying to read new lines
17:40:25.519 [debug] EOF reached, waiting before trying to read new lines
17:40:26.520 [debug] EOF reached, waiting before trying to read new lines
17:40:27.521 [debug] FileFollowerServer Handler: Got media attributes: %{"filename" => "/downloads/Vox Machina/Season 1/s01e00 youtube video #qn8tLbXWAtc/youtube video #qn8tLbXWAtc.NA", "id" => "qn8tLbXWAtc", "original_url" => "https://www.youtube.com/watch?v=qn8tLbXWAtc", "playlist_index" => 75, "title" => "youtube video #qn8tLbXWAtc", "upload_date" => "20161207"}
17:40:27.522 [debug] QUERY OK source="sources" db=0.2ms idle=1747.3ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" IN (?)) [53]
17:40:27.522 [debug] QUERY OK source="sources" db=0.1ms idle=998.4ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:40:27.522 [debug] QUERY OK source="media_items" db=0.1ms idle=748.1ms
SELECT max(m0."upload_date_index") FROM "media_items" AS m0 WHERE (date(m0."uploaded_at") = date(?) AND (m0."source_id" = ?)) [~U[2016-12-07 00:00:00Z], 53]
17:40:27.523 [debug] EOF reached, waiting before trying to read new lines
17:40:28.523 [debug] EOF reached, waiting before trying to read new lines
17:40:29.524 [debug] EOF reached, waiting before trying to read new lines
17:40:30.525 [debug] EOF reached, waiting before trying to read new lines
17:40:31.526 [debug] EOF reached, waiting before trying to read new lines
17:40:32.527 [debug] EOF reached, waiting before trying to read new lines
17:40:33.528 [debug] EOF reached, waiting before trying to read new lines
17:40:34.529 [debug] EOF reached, waiting before trying to read new lines
17:40:35.530 [debug] EOF reached, waiting before trying to read new lines
17:40:36.531 [debug] EOF reached, waiting before trying to read new lines
17:40:37.532 [debug] EOF reached, waiting before trying to read new lines
17:40:38.533 [debug] EOF reached, waiting before trying to read new lines
17:40:39.534 [debug] EOF reached, waiting before trying to read new lines
17:40:40.535 [debug] EOF reached, waiting before trying to read new lines
17:40:41.536 [debug] EOF reached, waiting before trying to read new lines
17:40:42.537 [debug] EOF reached, waiting before trying to read new lines
17:40:43.538 [debug] EOF reached, waiting before trying to read new lines
17:40:44.539 [debug] EOF reached, waiting before trying to read new lines
17:40:45.540 [debug] EOF reached, waiting before trying to read new lines
17:40:46.541 [debug] EOF reached, waiting before trying to read new lines
17:40:47.542 [debug] EOF reached, waiting before trying to read new lines
17:40:48.543 [debug] EOF reached, waiting before trying to read new lines
17:40:49.544 [debug] EOF reached, waiting before trying to read new lines
17:40:50.545 [debug] EOF reached, waiting before trying to read new lines
17:40:51.546 [debug] EOF reached, waiting before trying to read new lines
17:40:52.547 [debug] EOF reached, waiting before trying to read new lines
17:40:53.548 [debug] EOF reached, waiting before trying to read new lines
17:40:54.549 [debug] FileFollowerServer Handler: Got media attributes: %{"filename" => "/downloads/Vox Machina/Season 1/s01e00 youtube video #UHUBmvRNrik/youtube video #UHUBmvRNrik.NA", "id" => "UHUBmvRNrik", "original_url" => "https://www.youtube.com/watch?v=UHUBmvRNrik", "playlist_index" => 76, "title" => "youtube video #UHUBmvRNrik", "upload_date" => "20161214"}
17:40:54.550 [debug] QUERY OK source="sources" db=0.4ms idle=1775.3ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" IN (?)) [53]
17:40:54.550 [debug] QUERY OK source="sources" db=0.1ms idle=1776.0ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:40:54.551 [debug] QUERY OK source="media_items" db=0.1ms idle=1776.4ms
SELECT max(m0."upload_date_index") FROM "media_items" AS m0 WHERE (date(m0."uploaded_at") = date(?) AND (m0."source_id" = ?)) [~U[2016-12-14 00:00:00Z], 53]
17:40:54.551 [debug] EOF reached, waiting before trying to read new lines
17:40:55.552 [debug] EOF reached, waiting before trying to read new lines
17:40:56.553 [debug] EOF reached, waiting before trying to read new lines
17:40:57.554 [debug] EOF reached, waiting before trying to read new lines
17:40:58.555 [debug] EOF reached, waiting before trying to read new lines
17:40:59.556 [debug] EOF reached, waiting before trying to read new lines
17:41:00.557 [debug] EOF reached, waiting before trying to read new lines
17:41:01.558 [debug] EOF reached, waiting before trying to read new lines
17:41:02.559 [debug] EOF reached, waiting before trying to read new lines
17:41:03.560 [debug] EOF reached, waiting before trying to read new lines
17:41:04.561 [debug] EOF reached, waiting before trying to read new lines
17:41:05.562 [debug] EOF reached, waiting before trying to read new lines
17:41:06.563 [debug] EOF reached, waiting before trying to read new lines
17:41:07.564 [debug] EOF reached, waiting before trying to read new lines
17:41:08.565 [debug] EOF reached, waiting before trying to read new lines
17:41:09.566 [debug] EOF reached, waiting before trying to read new lines
17:41:09.574 [info] {"args":{"id":18},"id":45123,"meta":{},"system_time":1744753269574811751,"max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","event":"job:start","attempt":1,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:09.578 [debug] QUERY OK source="tasks" db=3.1ms idle=1800.4ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:09.578 [debug] QUERY OK source="sources" db=3.2ms idle=1800.3ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [18]
17:41:09.581 [debug] QUERY OK source="settings" db=2.4ms idle=1011.6ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:09.581 [debug] QUERY OK source="media_profiles" db=0.2ms idle=6.3ms
SELECT m0."id", m0."name", m0."redownload_delay_days", m0."output_path_template", m0."download_subs", m0."download_auto_subs", m0."embed_subs", m0."sub_langs", m0."download_thumbnail", m0."embed_thumbnail", m0."download_source_images", m0."download_metadata", m0."embed_metadata", m0."download_nfo", m0."sponsorblock_behaviour", m0."sponsorblock_categories", m0."shorts_behaviour", m0."livestream_behaviour", m0."audio_track", m0."preferred_resolution", m0."media_container", m0."marked_for_deletion_at", m0."inserted_at", m0."updated_at", m0."id" FROM "media_profiles" AS m0 WHERE (m0."id" = ?) [1]
17:41:09.581 [debug] QUERY OK source="sources" db=2.9ms idle=10.3ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" IN (?,?)) [53, 18]
17:41:09.582 [debug] QUERY OK source="settings" db=0.7ms idle=3.2ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:09.582 [debug] Fetching recent media IDs from YouTube RSS feed for source: UCjLF2fPUCooAVMgyAOVrmzg
17:41:10.220 [debug] Media ids fetched from RSS: ["OtHzr_gW8JI", "6FkqMlpPVvo", "-sIxEMJDzlY", "cfSV8Ggg1l4", "VWEp_kdzeXw", "o5lGT3SEtw4", "aZ-OX-TPxlk", "fzR2XqWgiI8", "Sf03_igsXjg", "A_rAPRABllw", "KmomHKhfkrQ", "xqRDDWakH0I", "2DNnD24wwXY", "ugvkglcupZY", "Rlt-i8dks2I"]
17:41:10.221 [debug] QUERY OK source="media_items" db=0.5ms idle=642.3ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 WHERE ((m0."source_id" = ?) AND m0."media_id" IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)) [18, "OtHzr_gW8JI", "6FkqMlpPVvo", "-sIxEMJDzlY", "cfSV8Ggg1l4", "VWEp_kdzeXw", "o5lGT3SEtw4", "aZ-OX-TPxlk", "fzR2XqWgiI8", "Sf03_igsXjg", "A_rAPRABllw", "KmomHKhfkrQ", "xqRDDWakH0I", "2DNnD24wwXY", "ugvkglcupZY", "Rlt-i8dks2I"]
17:41:10.222 [debug] QUERY OK source="media_items" db=0.5ms idle=640.7ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 INNER JOIN "sources" AS s1 ON s1."id" = m0."source_id" INNER JOIN "media_profiles" AS m2 ON m2."id" = s1."media_profile_id" WHERE ((m0."source_id" = ?) AND (((((NOT (NOT (m0."media_filepath" IS NULL)) AND NOT (m0."prevent_download" = 1)) AND ((s1."download_cutoff_date" IS NULL) OR date(m0."uploaded_at") >= s1."download_cutoff_date")) AND CASE
WHEN shorts_behaviour = 'only' AND livestream_behaviour = 'only' THEN
livestream = true OR short_form_content = true
WHEN shorts_behaviour = 'only' THEN
short_form_content = true
WHEN livestream_behaviour = 'only' THEN
livestream = true
WHEN shorts_behaviour = 'exclude' AND livestream_behaviour = 'exclude' THEN
short_form_content = false AND livestream = false
WHEN shorts_behaviour = 'exclude' THEN
short_form_content = false
WHEN livestream_behaviour = 'exclude' THEN
livestream = false
ELSE
true
END
) AND ((s1."title_filter_regex" IS NULL) OR regexp_like(m0."title", s1."title_filter_regex"))) AND (((s1."min_duration_seconds" IS NULL) OR duration_seconds >= s1."min_duration_seconds") AND ((s1."max_duration_seconds" IS NULL) OR duration_seconds <= s1."max_duration_seconds")))) [18]
17:41:10.223 [debug] QUERY OK source="tasks" db=0.1ms idle=641.2ms
INSERT INTO "tasks" ("job_id","source_id","inserted_at","updated_at") VALUES (?,?,?,?) RETURNING "id" [45133, 18, ~U[2025-04-15 21:41:10Z], ~U[2025-04-15 21:41:10Z]]
17:41:10.224 [info] {"args":{"id":18},"id":45123,"meta":{},"state":"success","max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","duration":648988,"event":"job:stop","attempt":1,"queue_time":19655,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:10.224 [debug] QUERY OK source="tasks" db=0.2ms idle=1.9ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:10.224 [debug] QUERY OK source="sources" db=0.1ms idle=1.5ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:41:10.567 [debug] EOF reached, waiting before trying to read new lines
17:41:10.575 [info] {"args":{"id":44},"id":45124,"meta":{},"system_time":1744753270575256108,"max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","event":"job:start","attempt":1,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:10.575 [info] {"args":{"id":46},"id":45125,"meta":{},"system_time":1744753270575301369,"max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","event":"job:start","attempt":1,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:10.576 [debug] QUERY OK source="sources" db=0.6ms queue=0.1ms idle=350.8ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [44]
17:41:10.576 [debug] QUERY OK source="sources" db=0.5ms idle=344.9ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [46]
17:41:10.576 [debug] QUERY OK source="tasks" db=0.9ms idle=350.6ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:10.576 [debug] QUERY OK source="settings" db=0.2ms idle=7.1ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:10.577 [debug] QUERY OK source="settings" db=0.4ms idle=1.3ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:10.577 [debug] QUERY OK source="media_profiles" db=0.4ms idle=0.5ms
SELECT m0."id", m0."name", m0."redownload_delay_days", m0."output_path_template", m0."download_subs", m0."download_auto_subs", m0."embed_subs", m0."sub_langs", m0."download_thumbnail", m0."embed_thumbnail", m0."download_source_images", m0."download_metadata", m0."embed_metadata", m0."download_nfo", m0."sponsorblock_behaviour", m0."sponsorblock_categories", m0."shorts_behaviour", m0."livestream_behaviour", m0."audio_track", m0."preferred_resolution", m0."media_container", m0."marked_for_deletion_at", m0."inserted_at", m0."updated_at", m0."id" FROM "media_profiles" AS m0 WHERE (m0."id" = ?) [3]
17:41:10.577 [debug] QUERY OK source="sources" db=0.5ms idle=0.4ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" IN (?,?,?)) [53, 46, 44]
17:41:10.577 [debug] QUERY OK source="media_profiles" db=0.3ms idle=0.5ms
SELECT m0."id", m0."name", m0."redownload_delay_days", m0."output_path_template", m0."download_subs", m0."download_auto_subs", m0."embed_subs", m0."sub_langs", m0."download_thumbnail", m0."embed_thumbnail", m0."download_source_images", m0."download_metadata", m0."embed_metadata", m0."download_nfo", m0."sponsorblock_behaviour", m0."sponsorblock_categories", m0."shorts_behaviour", m0."livestream_behaviour", m0."audio_track", m0."preferred_resolution", m0."media_container", m0."marked_for_deletion_at", m0."inserted_at", m0."updated_at", m0."id" FROM "media_profiles" AS m0 WHERE (m0."id" = ?) [3]
17:41:10.577 [debug] QUERY OK source="settings" db=0.1ms idle=0.7ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:10.577 [debug] Fetching recent media IDs from YouTube RSS feed for source: PLlUk42GiU2guNzWBzxn7hs8MaV7ELLCP_
17:41:10.577 [debug] QUERY OK source="settings" db=0.2ms idle=0.5ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:10.577 [debug] Fetching recent media IDs from YouTube RSS feed for source: UCl1a1FOUwxh5pOsGOZw37Cg
17:41:10.578 [debug] QUERY OK source="tasks" db=0.3ms queue=0.2ms idle=0.9ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:10.579 [debug] QUERY OK source="sources" db=0.1ms idle=1.8ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" IN (?,?,?)) [53, 46, 44]
17:41:11.045 [debug] Media ids fetched from RSS: ["e-YZvLUXcR8", "zTD2RZz6mlo", "1K8hGhpQzKg", "jvQXGs8IVBM", "0K9mKpAMREU", "Ama2wMxq84o", "nTbD1GJUFC0", "Uw-j-vjEAAo", "EHUCi6ZbVxU", "aO0HMmrZ4xs", "b5pz46kEhIM", "LQsJSqn71Fw", "EkXMxiAGUWg", "8Q8bVPpc84A", "DgPhiLBW7jo"]
17:41:11.045 [debug] QUERY OK source="media_items" db=0.4ms idle=467.8ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 WHERE ((m0."source_id" = ?) AND m0."media_id" IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)) [44, "e-YZvLUXcR8", "zTD2RZz6mlo", "1K8hGhpQzKg", "jvQXGs8IVBM", "0K9mKpAMREU", "Ama2wMxq84o", "nTbD1GJUFC0", "Uw-j-vjEAAo", "EHUCi6ZbVxU", "aO0HMmrZ4xs", "b5pz46kEhIM", "LQsJSqn71Fw", "EkXMxiAGUWg", "8Q8bVPpc84A", "DgPhiLBW7jo"]
17:41:11.047 [debug] QUERY OK source="media_items" db=0.4ms idle=468.8ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 INNER JOIN "sources" AS s1 ON s1."id" = m0."source_id" INNER JOIN "media_profiles" AS m2 ON m2."id" = s1."media_profile_id" WHERE ((m0."source_id" = ?) AND (((((NOT (NOT (m0."media_filepath" IS NULL)) AND NOT (m0."prevent_download" = 1)) AND ((s1."download_cutoff_date" IS NULL) OR date(m0."uploaded_at") >= s1."download_cutoff_date")) AND CASE
WHEN shorts_behaviour = 'only' AND livestream_behaviour = 'only' THEN
livestream = true OR short_form_content = true
WHEN shorts_behaviour = 'only' THEN
short_form_content = true
WHEN livestream_behaviour = 'only' THEN
livestream = true
WHEN shorts_behaviour = 'exclude' AND livestream_behaviour = 'exclude' THEN
short_form_content = false AND livestream = false
WHEN shorts_behaviour = 'exclude' THEN
short_form_content = false
WHEN livestream_behaviour = 'exclude' THEN
livestream = false
ELSE
true
END
) AND ((s1."title_filter_regex" IS NULL) OR regexp_like(m0."title", s1."title_filter_regex"))) AND (((s1."min_duration_seconds" IS NULL) OR duration_seconds >= s1."min_duration_seconds") AND ((s1."max_duration_seconds" IS NULL) OR duration_seconds <= s1."max_duration_seconds")))) [44]
17:41:11.048 [debug] QUERY OK source="tasks" db=0.2ms idle=468.6ms
INSERT INTO "tasks" ("job_id","source_id","inserted_at","updated_at") VALUES (?,?,?,?) RETURNING "id" [45134, 44, ~U[2025-04-15 21:41:11Z], ~U[2025-04-15 21:41:11Z]]
17:41:11.048 [info] {"args":{"id":44},"id":45124,"meta":{},"state":"success","max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","duration":473248,"event":"job:stop","attempt":1,"queue_time":896778,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:11.049 [debug] QUERY OK source="tasks" db=0.3ms idle=2.1ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:11.050 [debug] QUERY OK source="sources" db=0.1ms idle=1.9ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" IN (?,?)) [53, 46]
17:41:11.292 [debug] Media ids fetched from RSS: ["HVDgFjQVkpQ", "H4jVzo4-qac", "POZJGCrmuEM", "rh5Kx1N8aFQ", "RY7qp3_lFPw", "d7vKx0q0E4A", "IE46LSe8BUc", "mIOhA2bdYVo", "bnJV4NfVOmE", "WOsav24eTng", "A-hzVhslyMo", "c3IdcHwMZ_k", "BA2aFJM4lx4", "Bs_bkn8dhLs", "ltuJ8bqgsLk"]
17:41:11.293 [debug] QUERY OK source="media_items" db=0.5ms idle=244.4ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 WHERE ((m0."source_id" = ?) AND m0."media_id" IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)) [46, "HVDgFjQVkpQ", "H4jVzo4-qac", "POZJGCrmuEM", "rh5Kx1N8aFQ", "RY7qp3_lFPw", "d7vKx0q0E4A", "IE46LSe8BUc", "mIOhA2bdYVo", "bnJV4NfVOmE", "WOsav24eTng", "A-hzVhslyMo", "c3IdcHwMZ_k", "BA2aFJM4lx4", "Bs_bkn8dhLs", "ltuJ8bqgsLk"]
17:41:11.295 [debug] QUERY OK source="media_items" db=1.2ms idle=245.3ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 INNER JOIN "sources" AS s1 ON s1."id" = m0."source_id" INNER JOIN "media_profiles" AS m2 ON m2."id" = s1."media_profile_id" WHERE ((m0."source_id" = ?) AND (((((NOT (NOT (m0."media_filepath" IS NULL)) AND NOT (m0."prevent_download" = 1)) AND ((s1."download_cutoff_date" IS NULL) OR date(m0."uploaded_at") >= s1."download_cutoff_date")) AND CASE
WHEN shorts_behaviour = 'only' AND livestream_behaviour = 'only' THEN
livestream = true OR short_form_content = true
WHEN shorts_behaviour = 'only' THEN
short_form_content = true
WHEN livestream_behaviour = 'only' THEN
livestream = true
WHEN shorts_behaviour = 'exclude' AND livestream_behaviour = 'exclude' THEN
short_form_content = false AND livestream = false
WHEN shorts_behaviour = 'exclude' THEN
short_form_content = false
WHEN livestream_behaviour = 'exclude' THEN
livestream = false
ELSE
true
END
) AND ((s1."title_filter_regex" IS NULL) OR regexp_like(m0."title", s1."title_filter_regex"))) AND (((s1."min_duration_seconds" IS NULL) OR duration_seconds >= s1."min_duration_seconds") AND ((s1."max_duration_seconds" IS NULL) OR duration_seconds <= s1."max_duration_seconds")))) [46]
17:41:11.296 [debug] QUERY OK source="tasks" db=0.1ms idle=241.8ms
INSERT INTO "tasks" ("job_id","source_id","inserted_at","updated_at") VALUES (?,?,?,?) RETURNING "id" [45135, 46, ~U[2025-04-15 21:41:11Z], ~U[2025-04-15 21:41:11Z]]
17:41:11.297 [info] {"args":{"id":46},"id":45125,"meta":{},"state":"success","max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","duration":721488,"event":"job:stop","attempt":1,"queue_time":390558,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:11.297 [debug] QUERY OK source="tasks" db=0.4ms idle=1.9ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:11.298 [debug] QUERY OK source="sources" db=0.2ms idle=1.7ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:41:11.568 [debug] EOF reached, waiting before trying to read new lines
17:41:12.569 [debug] EOF reached, waiting before trying to read new lines
17:41:12.578 [info] {"args":{"id":47},"id":45126,"meta":{},"system_time":1744753272577947429,"max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","event":"job:start","attempt":1,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:12.578 [debug] QUERY OK source="sources" db=0.3ms queue=0.1ms idle=1280.0ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [47]
17:41:12.579 [debug] QUERY OK source="settings" db=0.1ms idle=1008.7ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:12.579 [debug] QUERY OK source="tasks" db=0.8ms idle=1275.3ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:12.579 [debug] QUERY OK source="media_profiles" db=0.3ms idle=7.8ms
SELECT m0."id", m0."name", m0."redownload_delay_days", m0."output_path_template", m0."download_subs", m0."download_auto_subs", m0."embed_subs", m0."sub_langs", m0."download_thumbnail", m0."embed_thumbnail", m0."download_source_images", m0."download_metadata", m0."embed_metadata", m0."download_nfo", m0."sponsorblock_behaviour", m0."sponsorblock_categories", m0."shorts_behaviour", m0."livestream_behaviour", m0."audio_track", m0."preferred_resolution", m0."media_container", m0."marked_for_deletion_at", m0."inserted_at", m0."updated_at", m0."id" FROM "media_profiles" AS m0 WHERE (m0."id" = ?) [3]
17:41:12.579 [debug] QUERY OK source="sources" db=0.5ms idle=1.5ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" IN (?,?)) [53, 47]
17:41:12.580 [debug] QUERY OK source="settings" db=0.8ms idle=1.0ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:12.580 [debug] Fetching recent media IDs from YouTube RSS feed for source: UCHkYrJ2Fbe7pBjEZvkFzi3A
17:41:12.819 [debug] Media ids fetched from RSS: ["o2YttS8udr4", "siLU0fJrtAU", "6zksM1aVeMc", "gz38zK4QTTc", "396RF645ThE", "8U6GFRcQrwo", "OtS8fY2s994", "yMa5EdV5zzY", "dRTZvT8tDDY", "PuBTKwyabyQ", "-_Shkyl26MU", "keR6MfbDHNc", "dHn474OhP-U", "KS-418px5Gs", "SpKMnWtzx38"]
17:41:12.820 [debug] QUERY OK source="media_items" db=0.3ms idle=240.6ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 WHERE ((m0."source_id" = ?) AND m0."media_id" IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)) [47, "o2YttS8udr4", "siLU0fJrtAU", "6zksM1aVeMc", "gz38zK4QTTc", "396RF645ThE", "8U6GFRcQrwo", "OtS8fY2s994", "yMa5EdV5zzY", "dRTZvT8tDDY", "PuBTKwyabyQ", "-_Shkyl26MU", "keR6MfbDHNc", "dHn474OhP-U", "KS-418px5Gs", "SpKMnWtzx38"]
17:41:12.821 [debug] QUERY OK source="media_items" db=0.5ms idle=241.5ms
SELECT m0."id", m0."uuid", m0."title", m0."media_id", m0."description", m0."original_url", m0."livestream", m0."short_form_content", m0."media_downloaded_at", m0."media_redownloaded_at", m0."uploaded_at", m0."upload_date_index", m0."duration_seconds", m0."playlist_index", m0."predicted_media_filepath", m0."media_filepath", m0."media_size_bytes", m0."thumbnail_filepath", m0."metadata_filepath", m0."nfo_filepath", m0."subtitle_filepaths", m0."last_error", m0."prevent_download", m0."prevent_culling", m0."culled_at", m0."source_id", m0."inserted_at", m0."updated_at" FROM "media_items" AS m0 INNER JOIN "sources" AS s1 ON s1."id" = m0."source_id" INNER JOIN "media_profiles" AS m2 ON m2."id" = s1."media_profile_id" WHERE ((m0."source_id" = ?) AND (((((NOT (NOT (m0."media_filepath" IS NULL)) AND NOT (m0."prevent_download" = 1)) AND ((s1."download_cutoff_date" IS NULL) OR date(m0."uploaded_at") >= s1."download_cutoff_date")) AND CASE
WHEN shorts_behaviour = 'only' AND livestream_behaviour = 'only' THEN
livestream = true OR short_form_content = true
WHEN shorts_behaviour = 'only' THEN
short_form_content = true
WHEN livestream_behaviour = 'only' THEN
livestream = true
WHEN shorts_behaviour = 'exclude' AND livestream_behaviour = 'exclude' THEN
short_form_content = false AND livestream = false
WHEN shorts_behaviour = 'exclude' THEN
short_form_content = false
WHEN livestream_behaviour = 'exclude' THEN
livestream = false
ELSE
true
END
) AND ((s1."title_filter_regex" IS NULL) OR regexp_like(m0."title", s1."title_filter_regex"))) AND (((s1."min_duration_seconds" IS NULL) OR duration_seconds >= s1."min_duration_seconds") AND ((s1."max_duration_seconds" IS NULL) OR duration_seconds <= s1."max_duration_seconds")))) [47]
17:41:12.822 [debug] QUERY OK source="tasks" db=0.1ms idle=241.7ms
INSERT INTO "tasks" ("job_id","source_id","inserted_at","updated_at") VALUES (?,?,?,?) RETURNING "id" [45136, 47, ~U[2025-04-15 21:41:12Z], ~U[2025-04-15 21:41:12Z]]
17:41:12.822 [info] {"args":{"id":47},"id":45126,"meta":{},"state":"success","max_attempts":20,"queue":"fast_indexing","worker":"Pinchflat.FastIndexing.FastIndexingWorker","source":"oban","duration":244557,"event":"job:stop","attempt":1,"queue_time":385327,"tags":["media_source","fast_indexing","show_in_dashboard"]}
17:41:12.823 [debug] QUERY OK source="tasks" db=0.2ms idle=1.7ms
SELECT t0."id", t0."job_id", t0."source_id", t0."media_item_id", t0."inserted_at", t0."updated_at", o1."id", o1."state", o1."queue", o1."worker", o1."args", o1."meta", o1."tags", o1."errors", o1."attempt", o1."attempted_by", o1."max_attempts", o1."priority", o1."attempted_at", o1."cancelled_at", o1."completed_at", o1."discarded_at", o1."inserted_at", o1."scheduled_at" FROM "tasks" AS t0 LEFT OUTER JOIN "oban_jobs" AS o1 ON o1."id" = t0."job_id" WHERE (o1."state" IN (?)) AND (? IN (SELECT value FROM JSON_EACH(o1."tags"))) ORDER BY o1."attempted_at" DESC ["executing", "show_in_dashboard"]
17:41:12.823 [debug] QUERY OK source="sources" db=0.1ms idle=1.3ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at", s0."id" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:41:13.570 [debug] EOF reached, waiting before trying to read new lines
17:41:14.571 [debug] EOF reached, waiting before trying to read new lines
17:41:15.572 [debug] EOF reached, waiting before trying to read new lines
17:41:16.573 [debug] EOF reached, waiting before trying to read new lines
17:41:17.574 [debug] EOF reached, waiting before trying to read new lines
17:41:18.575 [debug] EOF reached, waiting before trying to read new lines
17:41:19.576 [debug] EOF reached, waiting before trying to read new lines
17:41:20.577 [debug] EOF reached, waiting before trying to read new lines
17:41:21.578 [debug] FileFollowerServer Handler: Got media attributes: %{"filename" => "/downloads/Vox Machina/Season 1/s01e00 youtube video #2U21dJrY6ZI/youtube video #2U21dJrY6ZI.NA", "id" => "2U21dJrY6ZI", "original_url" => "https://www.youtube.com/watch?v=2U21dJrY6ZI", "playlist_index" => 77, "title" => "youtube video #2U21dJrY6ZI", "upload_date" => "20161221"}
17:41:21.579 [debug] QUERY OK source="sources" db=0.3ms idle=998.9ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" IN (?)) [53]
17:41:21.579 [debug] QUERY OK source="sources" db=0.1ms idle=804.9ms
SELECT s0."id", s0."enabled", s0."uuid", s0."custom_name", s0."description", s0."collection_name", s0."collection_id", s0."collection_type", s0."index_frequency_minutes", s0."fast_index", s0."cookie_behaviour", s0."download_media", s0."last_indexed_at", s0."download_cutoff_date", s0."retention_period_days", s0."original_url", s0."title_filter_regex", s0."output_path_template_override", s0."min_duration_seconds", s0."max_duration_seconds", s0."series_directory", s0."nfo_filepath", s0."poster_filepath", s0."fanart_filepath", s0."banner_filepath", s0."marked_for_deletion_at", s0."media_profile_id", s0."inserted_at", s0."updated_at" FROM "sources" AS s0 WHERE (s0."id" = ?) [53]
17:41:21.580 [debug] QUERY OK source="media_items" db=0.1ms idle=805.3ms
SELECT max(m0."upload_date_index") FROM "media_items" AS m0 WHERE (date(m0."uploaded_at") = date(?) AND (m0."source_id" = ?)) [~U[2016-12-21 00:00:00Z], 53]
17:41:21.580 [debug] EOF reached, waiting before trying to read new lines
17:41:22.581 [debug] EOF reached, waiting before trying to read new lines
17:41:23.582 [debug] EOF reached, waiting before trying to read new lines
17:41:24.583 [debug] EOF reached, waiting before trying to read new lines
17:41:25.584 [debug] EOF reached, waiting before trying to read new lines
17:41:26.585 [debug] EOF reached, waiting before trying to read new lines
17:41:27.586 [debug] EOF reached, waiting before trying to read new lines
17:41:28.587 [debug] EOF reached, waiting before trying to read new lines
17:41:29.588 [debug] EOF reached, waiting before trying to read new lines
17:41:30.589 [debug] EOF reached, waiting before trying to read new lines
17:41:31.590 [debug] EOF reached, waiting before trying to read new lines
17:41:32.591 [debug] EOF reached, waiting before trying to read new lines
17:41:33.592 [debug] EOF reached, waiting before trying to read new lines
17:41:34.593 [debug] EOF reached, waiting before trying to read new lines
17:41:35.594 [debug] EOF reached, waiting before trying to read new lines
17:41:36.595 [debug] EOF reached, waiting before trying to read new lines
17:41:37.596 [debug] EOF reached, waiting before trying to read new lines
17:41:38.597 [debug] EOF reached, waiting before trying to read new lines
17:41:39.598 [debug] EOF reached, waiting before trying to read new lines
17:41:40.467 [info] GET /app_info
17:41:40.467 [debug] Processing with PinchflatWeb.Settings.SettingController.app_info/2
Parameters: %{}
Pipelines: [:browser]
17:41:40.468 [debug] QUERY OK source="settings" db=0.1ms idle=1693.3ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:40.468 [debug] QUERY OK source="settings" db=0.1ms idle=1693.7ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:40.469 [debug] QUERY OK source="settings" db=0.1ms idle=1694.2ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:40.469 [debug] QUERY OK source="settings" db=0.1ms idle=869.5ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:40.469 [debug] QUERY OK source="settings" db=0.0ms idle=695.0ms
SELECT s0."id", s0."onboarding", s0."pro_enabled", s0."yt_dlp_version", s0."apprise_version", s0."apprise_server", s0."youtube_api_key", s0."route_token", s0."extractor_sleep_interval_seconds", s0."download_throughput_limit", s0."restrict_filenames", s0."video_codec_preference", s0."audio_codec_preference" FROM "settings" AS s0 LIMIT 1 []
17:41:40.470 [info] Sent 200 in 2ms
17:41:40.599 [debug] EOF reached, waiting before trying to read new lines
Add any other context about the problem here.
I tried changing various Media Profile settings to see if that would change anything, but it didn't.
As an update, it seems to have starting working again when my VPN switched to a US based IP.
I have noticed a pattern that when my VPN IP is another country it fails like this, but when it grabs a US based IP things seem to work fine.
EDIT to add: and its not throwing the "Confirm you are not a bot" error - It's just doing the above where it receives nothing.
Hmmm. Thanks for the context! I suspect that's something that's out of my hands but I'll see if I can repro at the very least
Same behavior without fast indexing enabled. Fast indexing works for me. IP doesn't seem to be banned. No VPN, US IP.
hey, i'm adding to this issue hoping to give more infos to the devs.
- App Version: 2025.3.17
- yt-dlp Version: 2025.02.19
- Apprise Version: v1.9.2
- System Architecture: x86_64-pc-linux-gnu
- Timezone: Europe/Rome
my compose looks like this:
pinchflat:
image: ghcr.io/kieraneglin/pinchflat:latest
container_name: pinchflat
user: ${PUID}:${PGID}
restart: unless-stopped
network_mode: service:gluetun
volumes:
- "${APPDATA}/pinchflat:/config:rw"
- "${MEDIA_FOLDER}/youtube:/downloads:rw"
environment:
- TZ=${TZ}
- BASIC_AUTH_USERNAME=${PINCHFLAT_USERNAME}
- BASIC_AUTH_PASSWORD=${PINCHFLAT_PASSWORD}
- EXPOSE_FEED_ENDPOINTS=true
i was running the container without the vpn, i tried switching to it today but with no luck.
i'm stuck on indexing sources just like other users here, but with a strange pattern. most of the sources are not indexed and continuously fail until the cap is reached, but some sources every now and then do happen to be correctly indexed. to be clear, this happened like 1 time every 3 weeks since march, but this should mean that my setup is still correct (or i hope so). ps: before march everything worked perfectly all the time.
i, like other users, have tried running the yt-dlp command in the container and it worked, here's the output:
docker exec -it pinchflat sh -c '/usr/local/bin/yt-dlp --simulate --skip-download -o "%(upload_date>%Y)s/%(title)s.%(ext)s" "https://youtube.com/@humansafari"'
[youtube:tab] Extracting URL: https://youtube.com/@humansafari
[youtube:tab] @humansafari: Downloading webpage
[youtube:tab] Downloading all uploads of the channel. To download only the videos in a specific tab, pass the tab's URL
[download] Downloading playlist: HumanSafari - Videos
[youtube:tab] UCB9-VTcQkAh-J7FpcinYj2A page 1: Downloading API JSON
[youtube:tab] UCB9-VTcQkAh-J7FpcinYj2A page 2: Downloading API JSON
[youtube:tab] UCB9-VTcQkAh-J7FpcinYj2A page 3: Downloading API JSON
[youtube:tab] UCB9-VTcQkAh-J7FpcinYj2A page 4: Downloading API JSON
[youtube:tab] Playlist HumanSafari - Videos: Downloading 120 items of 120
[download] Downloading item 1 of 120
[youtube] Extracting URL: https://www.youtube.com/watch?v=z9dgSbO9gaU
[youtube] z9dgSbO9gaU: Downloading webpage
[youtube] z9dgSbO9gaU: Downloading tv client config
[youtube] z9dgSbO9gaU: Downloading player b2858d36
[youtube] z9dgSbO9gaU: Downloading tv player API JSON
[youtube] z9dgSbO9gaU: Downloading ios player API JSON
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] z9dgSbO9gaU: nsig extraction failed: Some formats may be missing
n = M762LipMJS2La6wl ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] z9dgSbO9gaU: nsig extraction failed: Some formats may be missing
n = O2DmhbNbCvIH2JAA ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] z9dgSbO9gaU: nsig extraction failed: Some formats may be missing
n = lLnW534JFna49q57 ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
[youtube] z9dgSbO9gaU: Downloading m3u8 information
[info] z9dgSbO9gaU: Downloading 1 format(s): 625+234
[download] Downloading item 2 of 120
[youtube] Extracting URL: https://www.youtube.com/watch?v=DFeg2j7LSaY
[youtube] DFeg2j7LSaY: Downloading webpage
[youtube] DFeg2j7LSaY: Downloading tv client config
[youtube] DFeg2j7LSaY: Downloading tv player API JSON
[youtube] DFeg2j7LSaY: Downloading ios player API JSON
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] DFeg2j7LSaY: nsig extraction failed: Some formats may be missing
n = mrH2bDulmcY-r3MC ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] DFeg2j7LSaY: nsig extraction failed: Some formats may be missing
n = ic52iI0EOIXn-MKk ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] DFeg2j7LSaY: nsig extraction failed: Some formats may be missing
n = vhAWva1NeqAN6XxH ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
[youtube] DFeg2j7LSaY: Downloading m3u8 information
[info] DFeg2j7LSaY: Downloading 1 format(s): 625+234
[download] Downloading item 3 of 120
[youtube] Extracting URL: https://www.youtube.com/watch?v=2QSfl3q5gyI
[youtube] 2QSfl3q5gyI: Downloading webpage
[youtube] 2QSfl3q5gyI: Downloading tv client config
[youtube] 2QSfl3q5gyI: Downloading tv player API JSON
[youtube] 2QSfl3q5gyI: Downloading ios player API JSON
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] 2QSfl3q5gyI: nsig extraction failed: Some formats may be missing
n = 7vaWx1szd5TCd6DQ ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] 2QSfl3q5gyI: nsig extraction failed: Some formats may be missing
n = BXdmzqROOKPccPyT ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] Falling back to generic n function search
player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
WARNING: [youtube] 2QSfl3q5gyI: nsig extraction failed: Some formats may be missing
n = z3O2J3CtLzOsjWP1 ; player = https://www.youtube.com/s/player/b2858d36/player_ias.vflset/en_US/base.js
[youtube] 2QSfl3q5gyI: Downloading m3u8 information
[info] 2QSfl3q5gyI: Downloading 1 format(s): 625+234
[download] Downloading item 4 of 120
[youtube] Extracting URL: https://www.youtube.com/watch?v=sizfROHuIIU
[youtube] sizfROHuIIU: Downloading webpage
^C
ERROR: Interrupted by user
and this is the logs after pinchflat used the yt-dlp command:
17:12:03.999 [info] [command_wrapper]: /usr/local/bin/yt-dlp called with: https://youtube.com/@humansafari --simulate --skip-download --ignore-no-formats-error --no-warnings --output /downloads/HumanSafari/Season %(upload_date>%Y)S/s%(upload_date>%Y)Se%(upload_date>%m%d)S00 - %(title)S.%(ext)S --remux-video mp4 --format-sort res:720,+codec:av01:opus --format bestvideo+bestaudio[format_note*=original]/bestvideo*+bestaudio/best --print-to-file %(.{id,title,live_status,original_url,description,aspect_ratio,duration,upload_date,timestamp,playlist_index,filename})j /tmp/pinchflat/data/bd/82/bd82a4bb02bfe46d0d5c0b9c94a58da4b465bb425f76d5deb29dec8172f88485.json --sleep-requests 98 --sleep-interval 134 --sleep-subtitles 98 --windows-filenames --quiet --cache-dir /tmp/pinchflat/data/yt-dlp-cache
17:12:03.999 [info] [command_wrapper]: /usr/local/bin/yt-dlp called with: https://youtube.com/@kurzgesagt --simulate --skip-download --ignore-no-formats-error --no-warnings --output /downloads/Kurzgesagt – In a Nutshell/Season %(upload_date>%Y)S/s%(upload_date>%Y)Se%(upload_date>%m%d)S00 - %(title)S.%(ext)S --remux-video mp4 --format-sort res:720,+codec:av01:opus --format bestvideo+bestaudio[format_note*=original]/bestvideo*+bestaudio/best --print-to-file %(.{id,title,live_status,original_url,description,aspect_ratio,duration,upload_date,timestamp,playlist_index,filename})j /tmp/pinchflat/data/e1/d9/e1d939d02c1c9386802c061cc85b97b2dc7cb880d180c4bf906cffa14eefa5c6.json --sleep-requests 125 --sleep-interval 120 --sleep-subtitles 93 --windows-filenames --quiet --cache-dir /tmp/pinchflat/data/yt-dlp-cache
17:12:04.999 [debug] EOF reached, waiting before trying to read new lines
...
many lines of [debug] EOF reached, waiting before trying to read new lines
...
17:22:03.386 [debug] EOF reached, waiting before trying to read new lines
17:22:04.386 [debug] No activity for 600000ms. Requesting stop.
17:22:04.386 [debug] Gracefully stopping file follower
17:22:04.387 [debug] No activity for 600000ms. Requesting stop.
17:22:04.387 [debug] Gracefully stopping file follower
i'm open to test things on my setup if needed, and i hope that these infos are useful to you in some way!
