[Bug] Error while downloading

[Hiradpi]

Describe the bug

when im downloading multiple files i sometimes get this error

Error: group:                                                                                                                                                                                          
    github.com/gotd/td/mtproto.(*Conn).Run                                                                                                                                                             
        github.com/gotd/[email protected]/mtproto/conn.go:214                                                                                                                                                
  - task readLoop:                                                                                                                                                                                     
    github.com/gotd/td/tdsync.(*LogGroup).Go.func1                                                                                                                                                     
        github.com/gotd/[email protected]/tdsync/log_group.go:48                                                                                                                                             
  - read:                                                                                                                                                                                              
    github.com/gotd/td/mtproto.(*Conn).readLoop                                                                                                                                                        
        github.com/gotd/[email protected]/mtproto/read.go:193                                                                                                                                                
  - read:                                                                                                                                                                                              
    github.com/gotd/td/transport.(*connection).Recv                                                                                                                                                    
        github.com/gotd/[email protected]/transport/connection.go:70                                                                                                                                         
  - read intermediate:                                                                                                                                                                                 
    github.com/gotd/td/proto/codec.Intermediate.Read                                                                                                                                                   
        github.com/gotd/[email protected]/proto/codec/intermediate.go:73                                                                                                                                     
  - read length:                                                                                                                                                                                       
    github.com/gotd/td/proto/codec.readLen                                                                                                                                                             
        github.com/gotd/[email protected]/proto/codec/codec.go:35                                                                                                                                            
  - EOF

here is my command flag tdl dl -f tdl-export.json --desc

To Reproduce

export chat download exported chat

Expected behavior

it should not just die after getting an error

Version

Version: 0.18.5 Commit: 2321013 Date: 2025-01-25T11:11:37Z

go1.21.13 linux/amd64

Which OS are you running tdl on?

Windows

Additional context

No response

[mhdy2233]

I also encountered the same problem.

[shantanu561993]

Same here

[Ganim]

Its happens with me every time I download a lot of files, or more than one huge files (1gb +) at same time.

[Ender-Wang]

Same here (same crash log); I was downloading 1700+ files from a channel via an exported chat json and ran into this crash three times; I am glad when I rerun the same command, it says I got an unfinished download and can resume from there. But it resume way before it crashed, crashed at 1500/1700 or so, then it resumed from 800/1700.

Edit: I tried v0.18.3 and got the exact same crash log, but this time, it crashes at 1300/1700 (I started from the beginning).

[zhheo]

I also encountered the same problem. add --reconnect-timeout 0 solve it.

[KNaiFen]

I also encountered the same problem. add --reconnect-timeout 0 solve it.

good, it work

[eximoelle]

Hey, guys!

I've also encountered this error. It happens at different times after starting the download — after 10-20 minutes.

I investigated the problem with different parameters and came to the following results.

If I reduce the number of threads and simultaneous downloads to 1 — the task runs continuously for more than 8 hours without failures.

Any other combination of these parameters greater than 1 will result in an error.

So, the workaround is

TDL_THREADS="1"

TDL_LIMIT="1"

Side effect: very slow download. But stable. I hope this information will help the developer to find a solution.

So, I read here that setting connect timeout to infinity helped someone. I will try this solution and add more later.

---

So, after a day of lazy testing came to these conclusions:

• the proposed above reduction of threads and limit does not prevent this error completely, but only postpones the moment of the error

• the above suggested increase of reconnection time to infinity not only does not solve the problem, but also gives another error - segmentation fault (in the attached log)

segmentation error tdl.txt

As a result, I came to the decision to write a script that will wait until the process terminates with an error and restart it again.

If anyone is interested, here is its content:

#!/bin/bash

# Путь к файлу и директории
FILE=~/Downloads/tdl-export.json
DEST="/Volumes/..."

# Бесконечный цикл с перезапуском команды
while true; do
    echo "Запуск tdl download..."
    tdl download -f "$FILE" -d "$DEST" --continue --skip-same

    EXIT_CODE=$?

    echo "tdl завершился с кодом $EXIT_CODE"

    if [ $EXIT_CODE -eq 0 ]; then
        echo "Завершено без ошибок. Выход."
        break
    fi

    echo "Обнаружена ошибка. Перезапуск через 10 секунд..."
    sleep 10
done

I would like to thank the developer for his work and can offer my help in testing hypotheses or solutions if he is interested.

[CodingMoeButa]

Same here:

Error: group:
    github.com/gotd/td/mtproto.(*Conn).Run
        github.com/gotd/[email protected]/mtproto/conn.go:214
  - task readLoop:
    github.com/gotd/td/tdsync.(*LogGroup).Go.func1
        github.com/gotd/[email protected]/tdsync/log_group.go:48
  - read:
    github.com/gotd/td/mtproto.(*Conn).readLoop
        github.com/gotd/[email protected]/mtproto/read.go:193
  - read:
    github.com/gotd/td/transport.(*connection).Recv
        github.com/gotd/[email protected]/transport/connection.go:70
  - read intermediate:
    github.com/gotd/td/proto/codec.Intermediate.Read
        github.com/gotd/[email protected]/proto/codec/intermediate.go:73
  - read length:
    github.com/gotd/td/proto/codec.readLen
        github.com/gotd/[email protected]/proto/codec/codec.go:35
  - EOF