yt-dlp
yt-dlp copied to clipboard
yt-dlp
Douyin always errors saying fresh cookies are needed, even when passing them
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
- [X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field
Checklist
- [X] I'm reporting that yt-dlp is broken on a supported site
- [X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the guidelines for opening an issue
- [ ] I've read about sharing account credentials and I'm willing to share it if required
Region
China
Provide a description that is worded well enough to be understood
yt-dlp --verbose --cookies douyin.com_cookies.txt -o "摄影师今个没鸡腿晚风很温柔_7356265095304875264.%(ext)s" https://www.douyin.com/video/7356265095304875264 [debug] Command-line config: ['--verbose', '--cookies', 'douyin.com_cookies.txt', '-o', '摄影师今个没鸡腿晚风很温柔_7356265095304875264.%(ext)s', 'https://www.douyin.com/video/7356265095304875264'] [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (pip) [debug] Python 3.11.8 (CPython AMD64 64bit) - Windows-10-10.0.19041-SP0 (OpenSSL 3.0.13 30 Jan 2024) [debug] exe versions: ffmpeg 2022-01-27-git-3c831847a8-full_build-www.gyan.dev (setts), ffprobe 2022-01-27-git-3c831847a8-full_build-www.gyan.dev [debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2024.02.02, mutagen-1.46.0, requests-2.31.0, sqlite3-3.41.2, urllib3-1.26.18, websockets-12.0 [debug] Proxy map: {} [debug] Request Handlers: urllib, requests, websockets [debug] Loaded 1810 extractors [Douyin] Extracting URL: https://www.douyin.com/video/7356265095304875264 [Douyin] 7356265095304875264: Downloading web detail JSON WARNING: [Douyin] 7356265095304875264: Failed to parse JSON: Expecting value in '': line 1 column 1 (char 0) ERROR: [Douyin] 7356265095304875264: Fresh cookies (not necessarily logged in) are needed; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U File "C:\Users\cyy.conda\envs\working\Lib\site-packages\yt_dlp\extractor\common.py", line 734, in extract ie_result = self._real_extract(url) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\cyy.conda\envs\working\Lib\site-packages\yt_dlp\extractor\tiktok.py", line 1135, in _real_extract raise ExtractorError(
Provide verbose output that clearly demonstrates the problem
- [X] Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - [ ] If using API, add
'verbose': TruetoYoutubeDLparams instead - [X] Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
yt-dlp --verbose --cookies douyin.com_cookies.txt -o "摄影师今个没鸡腿晚风很温柔_7356265095304875264.%(ext)s" https://www.douyin.com/video/7356265095304875264
[debug] Command-line config: ['--verbose', '--cookies', 'douyin.com_cookies.txt', '-o', '摄影师今个没鸡腿晚风很温柔_7356265095304875264.%(ext)s', 'https://www.douyin.com/video/7356265095304875264']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (pip)
[debug] Python 3.11.8 (CPython AMD64 64bit) - Windows-10-10.0.19041-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2022-01-27-git-3c831847a8-full_build-www.gyan.dev (setts), ffprobe 2022-01-27-git-3c831847a8-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2024.02.02, mutagen-1.46.0, requests-2.31.0, sqlite3-3.41.2, urllib3-1.26.18, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[Douyin] Extracting URL: https://www.douyin.com/video/7356265095304875264
[Douyin] 7356265095304875264: Downloading web detail JSON
WARNING: [Douyin] 7356265095304875264: Failed to parse JSON: Expecting value in '': line 1 column 1 (char 0)
ERROR: [Douyin] 7356265095304875264: Fresh cookies (not necessarily logged in) are needed; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "C:\Users\cyy\.conda\envs\working\Lib\site-packages\yt_dlp\extractor\common.py", line 734, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\cyy\.conda\envs\working\Lib\site-packages\yt_dlp\extractor\tiktok.py", line 1135, in _real_extract
raise ExtractorError(
If the video isn't private, try using logged-out cookies (not from any account). Open the video link in browser, solve the catpcha. and export/extract those browser cookies
Hi! My region is the USA and I have a Douyin account. When I try using the Douyin extractor both logged out and logged in, I get the same error:
yt-dlp -vU --cookies-from-browser chrome https://www.douyin.com/video/7360255988722650409 --verbose
[debug] Command-line config: ['-vU', '--cookies-from-browser', 'chrome', 'https://www.douyin.com/video/7360255988722650409', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (pip)
[debug] Python 3.12.3 (CPython arm64 64bit) - macOS-14.1.1-arm64-arm-64bit (OpenSSL 3.2.1 30 Jan 2024)
[debug] exe versions: ffmpeg 3.4.1, ffprobe 3.4.1
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.2, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
Extracting cookies from chrome
[debug] Extracting cookies from: "/Users/<user>/Library/Application Support/Google/Chrome/Default/Cookies"
[debug] using find-generic-password to obtain password from OSX keychain
Extracted 3088 cookies from chrome
[debug] cookie version breakdown: {'v10': 2994, 'other': 0, 'unencrypted': 94}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: [email protected] from yt-dlp/yt-dlp
yt-dlp is up to date ([email protected] from yt-dlp/yt-dlp)
[Douyin] Extracting URL: https://www.douyin.com/video/7360255988722650409
[Douyin] 7360255988722650409: Downloading web detail JSON
WARNING: [Douyin] 7360255988722650409: Failed to parse JSON: Expecting value in '': line 1 column 1 (char 0)
ERROR: [Douyin] 7360255988722650409: Fresh cookies (not necessarily logged in) are needed; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/opt/homebrew/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 734, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/tiktok.py", line 1135, in _real_extract
raise ExtractorError(
I've tried other browsers as well, such as Edge, Opera, and Firefox, and they all throw the same error. I know projects like F2 make API calls to Douyin's API endpoints, but I've not had a chance to look at yt-dlp's version.
same error warning message, logged-in / logged-out cookies get this same error: Failed to parse JSON: Expecting value in '': line 1 column 1 (char 0)
same error warning message, logged-in / logged-out cookies get this same error:
WARNING: [Douyin] 7364022131870453028: Failed to parse JSON: Expecting value in '': line 1 column 1 (char 0) ERROR: [Douyin] 7364022131870453028: Fresh cookies (not necessarily logged in) are needed; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
Just did some research on this. I'm a web developer, not a python developer, let alone a contributor to yt-dlp. Anyways, I've found out the following.
When requesting the same URL (as yt-dlp does) directly in the Firefox developer console using the fetch method, the requests succeeds with a valid JSON response body. So for example calling in the console:
fetch("https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=<ID>")
However, when inspecting the network tab, a whole bunch of GET parameters are added to this request. They have the following keys:
verifyFp=<SOME TOKEN>
fp=<SAME TOKEN AS verifyFp>
msToken=<SOME RANDOM STRING>
a_bogus=<ANOTHER RANDOM STRING>
When doing the same fetch-call from the JS console those parameters stay the same, except for 'a_bogus'. When omitting one of those parameters OR sending the request with the same a_bogus value, the response returns a empty body. This is the error message people are reporting above.
So in short; I suspect they're using some sort of middleware for the fetch method, adding those parameters to the URL. Where a_bogus is generated uniquely for every request. (I didn't dig deeper in this, maybe later). So unless yt-dlp mimics this functionally, on top off using the cookies of a browser, the requests will fail.
Googling on a_bogus gave me this repo: https://github.com/jackluson/a_bogus_douyin which might be a clue, not sure.
Hope this helps some :)
I found a source code file that creates a valid a_bogus string from a chinese Douyin downloader: https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/crawlers/douyin/web/abogus.py
I found a source code file that creates a valid
a_bogusstring from a chinese Douyin downloader: https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/crawlers/douyin/web/abogus.py
finally, i do the download by myself instead of yt-dlp. I also use yt-dlp for youtube and bilibili
I found a source code file that creates a valid
a_bogusstring from a chinese Douyin downloader: https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/crawlers/douyin/web/abogus.pyfinally, i do the download by myself instead of yt-dlp. I also use yt-dlp for youtube and bilibili
can you tell me how you can fix it and can download videos on douyin?
