telegram_media_downloader icon indicating copy to clipboard operation
telegram_media_downloader copied to clipboard

update to 2.0.2,more error,Is it a network problem? The app is fast and stable

Open home2020 opened this issue 1 year ago • 10 comments

       ****python3.10.5
       use app to download is ok, 
       All channels are the same****

ERROR client.py:979 Traceback (most recent call last):
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/client.py", line 848, in get_file
r = await session.invoke(
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 381, in invoke
return await self.invoke(query, retries - 1, timeout)
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 381, in invoke
return await self.invoke(query, retries - 1, timeout)
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 381, in invoke
return await self.invoke(query, retries - 1, timeout)
[Previous line repeated 2 more times]
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 374, in invoke
raise e from None
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 361, in invoke
return await self.send(query, timeout=timeout)
File "/usr/local/python-3.10.5/lib/python3.10/site-packages/pyrogram/session/session.py", line 326, in send
raise TimeoutError
TimeoutError

for more imformation ##################################################################################################

[00:59:46] Telegram Media Downloader v2.0.2, meta.py:18 Copyright (C) 2019 Dineshkarthik https://github.com/Dineshkarthik
Licensed under the terms of the MIT License meta.py:21

[00:59:46] INFO Device: CPython 3.10.5 - Telegram Media Downloader 2.0.2 meta.py:22 INFO System: Linux 5.12.4-1.el7.elrepo.x86_64 (EN) meta.py:23 /root/telegram_media_downloader/media_downloader.py:360: DeprecationWarning: There is no current event loop updated_config = asyncio.get_event_loop().run_until_complete( [00:59:47] INFO Connecting... connection.py:56 INFO Connected! Production DC5 - IPv4 - TCPAbridgedO connection.py:63 INFO NetworkTask started session.py:271 INFO Time synced: 2022-07-30 16:59:47.440387 UTC msg_id.py:45 [00:59:48] INFO Session initialized: Layer 143 session.py:131 INFO Device: CPython 3.10.5 - Pyrogram 2.0.35 session.py:132 INFO System: Linux 5.12.4-1.el7.elrepo.x86_64 (EN) session.py:133 INFO Session started session.py:148 INFO PingTask started session.py:249 INFO Started 6 HandlerTasks dispatcher.py:154 [00:59:50] INFO Connecting... connection.py:56 [00:59:51] INFO Connected! Production DC5 (media) - IPv4 - TCPAbridgedO
INFO NetworkTask started
[00:59:53] INFO Session initialized: Layer 143
INFO Device: CPython 3.10.5 - Pyrogram 2.0.35
INFO System: Linux 5.12.4-1.el7.elrepo.x86_64 (EN)
INFO Session started
INFO PingTask started
INFO Start creating a new auth key on DC1
INFO Connecting...
INFO Connected! Production DC1 - IPv4 - TCPAbridgedO
[00:59:57] INFO Done auth key exchange: DhGenOk
INFO Disconnected
INFO Connecting...
[00:59:58] INFO Connected! Production DC1 (media) - IPv4 - TCPAbridgedO
INFO NetworkTask started
[00:59:59] INFO Disconnected
INFO NetworkTask stopped
INFO Session stopped
INFO Connecting...
INFO Connected! Production DC1 (media) - IPv4 - TCPAbridgedO

home2020 avatar Jul 30 '22 17:07 home2020

I have the same problem with timeout error. Got many 0 byte videos downloaded.

qk-li avatar Jul 30 '22 18:07 qk-li

same error here

cleitonpin avatar Aug 03 '22 21:08 cleitonpin

same error here too

RafidSadat404 avatar Aug 08 '22 16:08 RafidSadat404

I also have the same problem. Didn't see this issue logged before and created a duplicate discussion thread : https://github.com/Dineshkarthik/telegram_media_downloader/discussions/309

Benouzao avatar Aug 09 '22 05:08 Benouzao

I also meet this problem, and I add another size check to temporary fix it. As in my opinion,

  1. if download_path is None, it download fail;
  2. if downloaded file size is not same as telegram gived, it download fail;

and when download media fail, we should remove wrong local file, so we can re-run the script.

See my patch in: https://editor.mergely.com/PwqHsSL5/

195,196c195,206
<                         logger.info("Media downloaded - %s", download_path)
<                     DOWNLOADED_IDS.append(message.id)
---
>                         media_size = getattr(_media, 'file_size')
>                         if media_size is not None:
>                             download_size = os.path.getsize(download_path)
>                             if media_size == download_size:
>                                 logger.info("Media downloaded - %s", download_path)
>                                 DOWNLOADED_IDS.append(message.id)
>                             else:
>                                 logger.info('Media downloaded with wrong size - %s', download_path)
>                                 os.remove(download_path)
>                                 FAILED_IDS.append(message.id)
>                     else:
>                         FAILED_IDS.append(message.id)
376a387
> 

I also make a feature: delete suceessful download message , since it is very useful to me. See in https://editor.mergely.com/sC1eCvKR/

Rhilip avatar Aug 09 '22 13:08 Rhilip

I also meet this problem, and I add another size check to temporary fix it. As in my opinion,

  1. if download_path is None, it download fail;
  2. if downloaded file size is not same as telegram gived, it download fail; and when download media fail, we should remove wrong local file, so we can re-run the script.

See my patch in: https://editor.mergely.com/PwqHsSL5/

195,196c195,206
<                         logger.info("Media downloaded - %s", download_path)
<                     DOWNLOADED_IDS.append(message.id)
---
>                         media_size = getattr(_media, 'file_size')
>                         if media_size is not None:
>                             download_size = os.path.getsize(download_path)
>                             if media_size == download_size:
>                                 logger.info("Media downloaded - %s", download_path)
>                                 DOWNLOADED_IDS.append(message.id)
>                             else:
>                                 logger.info('Media downloaded with wrong size - %s', download_path)
>                                 os.remove(download_path)
>                                 FAILED_IDS.append(message.id)
>                     else:
>                         FAILED_IDS.append(message.id)
376a387
> 

I also make a feature: delete suceessful download message , since it is very useful to me. See in https://editor.mergely.com/sC1eCvKR/

It is very helpful, but it will still output the error log, and it will still report an error at the next startup. I don't know why

dream2333 avatar Aug 09 '22 15:08 dream2333

It is very helpful, but it will still output the error log, and it will still report an error at the next startup. I don't know why

Yes, So I wrote temporary, and you may run many times to full download all media. size check only help when You download lots of empty files, since current script think those are downloaded and not failed.

There still have some other changes might help you, ( But it's still a temporary solution. )

  1. in my last patch, change all FAILED_IDS.append(message.id) to raise TypeError('wrong size'), It will let script run into retry and will retry 3 times , in this case you can increase asyncio.sleep time at L220.

https://github.com/Dineshkarthik/telegram_media_downloader/blob/b0501a1e72b98aae5bb6ad0eca1161675e5731ef/media_downloader.py#L214-L226

  1. If you don't care about download speed, you can enable takeout session by pass takeout=True when init pyrogram Client. See docs in: https://docs.pyrogram.org/api/client#pyrogram.Client

https://github.com/Dineshkarthik/telegram_media_downloader/blob/b0501a1e72b98aae5bb6ad0eca1161675e5731ef/media_downloader.py#L305-L310


see my full patch compare: https://editor.mergely.com/4ZOOPvTx/

136a137
>     config,
195,196c196,210
<                         logger.info("Media downloaded - %s", download_path)
<                     DOWNLOADED_IDS.append(message.id)
---
>                         media_size = getattr(_media, 'file_size')
>                         if media_size is not None:
>                             download_size = os.path.getsize(download_path)
>                             if media_size == download_size:
>                                 logger.info("Media downloaded - %s", download_path)
>                                 DOWNLOADED_IDS.append(message.id)
>                                 
>                                 # Note: It will delete downloaded message, Please make sure privilege
>                                 await client.delete_messages(config["chat_id"], message.id)
>                                 logger.info('Message delete - %s', message.id)
>                                 
>                             else:
>                                 logger.info('Media downloaded with wrong size - %s', download_path)
>                                 os.remove(download_path)
>                                 raise TypeError('Media downloaded with wrong size')
217c231
<                 "Timeout Error occurred when downloading Message[%d], retrying after 5 seconds",
---
>                 "Timeout Error occurred when downloading Message[%d], retrying after 60 seconds",
220c234
<             await asyncio.sleep(5)
---
>             await asyncio.sleep(60)
240a255
>     config, 
276c291
<             download_media(client, message, media_types, file_formats)
---
>             download_media(config, client, message, media_types, file_formats)
309a325
>         takeout=True
332a349
>                 config,
344a362
>             config,

Rhilip avatar Aug 12 '22 12:08 Rhilip

@Rhilip

If you don't care about download speed, you can enable takeout session by pass takeout=True when init pyrogram Client. See docs in: https://docs.pyrogram.org/api/client#pyrogram.Client

Even though I'm an IT specialist, I'm not familiar with this. Can you please explain, how to enable takeout=True?

When using v1.5.1

Thanks and best regards, Brendan

BrendanMK avatar Aug 26 '22 08:08 BrendanMK

I used your modified *.py file from https://editor.mergely.com/4ZOOPvTx/ This is what I received from Telegram:

Data export request. Brendan, we received a request from your account to export your Telegram data.

*Why would your .py file need my permission to export my Telegram data!

BrendanMK avatar Aug 26 '22 13:08 BrendanMK

This issue has been automatically marked as stale because it has not had recent activity in the past 45 days. It will be closed if no further activity occurs in the next 7 days. Thank you for your contributions.

stale[bot] avatar Dec 26 '22 10:12 stale[bot]

PR?

SavSanta avatar Jan 02 '23 03:01 SavSanta

This issue has been automatically marked as stale because it has not had recent activity in the past 45 days. It will be closed if no further activity occurs in the next 7 days. Thank you for your contributions.

stale[bot] avatar Apr 02 '23 05:04 stale[bot]