lftp Upload hang at 100%

Upload hang at 100%

Open silvioto opened this issue 7 years ago • 12 comments

I have a script that upload my files with FTPS. It work well with small files, but on 70mb the transfer end with a message "100% - Waiting for transfer to complete". The file is entirely uploaded but connection stay open and go in timeout forever, and the script don't execute the following command.

I tryed to read manual but I don't have idea which options should I use to solve. Thanks for your help.

I'm using Debian Jessie package of lftp v. 4.6.0-1+deb8u1.

loglftp.txt toftps.txt

May 17 '17 11:05 silvioto

I'm frequently running into a similar, if not the same error. The transfer of a particular file succeeds (100% transfer status and the file is verifyably complete), but the server closes the connection to lftp and lftp hangs itself up in a reconnect loop.

Sep 17 '18 14:09 nausicaea

Apr 26 '19 18:04 jifang

Please include debug logs.

пт, 26 апр. 2019, 21:19 Ji Fang [email protected]:

+1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lavv17/lftp/issues/346#issuecomment-487153557, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHLWXDBSTEXMGS3PSW532TPSNBT7ANCNFSM4DLXBJLQ .

Apr 28 '19 11:04 lavv17

Apr 15 '20 20:04 aweidler

I stopped having the reconnect loop, but I still get connection terminations (but that is now client independent). For that, luckily, I managed to use the --loop flag with success.

Apr 16 '20 05:04 nausicaea

Nov 19 '20 15:11 mweichert

May 09 '21 14:05 ThirtySix361

I have same with version 4.8.4 or wirth 4.9.2

Is only happens i have force use TLS on my firtzbox.

Without TLS i have no issue

Jun 13 '21 13:06 tome9111991

@lavv17 Issue still with latest source code. Mybe issue with ftpes ?

---- using user `ftpuser' and password from ~/.netrc
---- Löse Hostadresse auf...
---- 1  Adresse gefunden: xx.xxx.xx.xxx
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate depth: 0; subject: /CN=my dyndns.de; issuer: /CN=my dyndns.de
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PWD
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 257 "/" is current directory.
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> PORT 62,171,163,52,197,25
---> STOR config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 38751 akzeptiert
<--- 200 PORT command sucessful.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 81920
---> PORT 62,171,163,52,234,235
---> REST 81920
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 81920 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 44871 akzeptiert
<--- 350 Restarting at 81920. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 131072
---> PORT 62,171,163,52,198,101
---> REST 131072
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 131072 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 39489 akzeptiert
<--- 350 Restarting at 131072. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 196608
---> PORT 62,171,163,52,221,77
---> REST 196608
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 196608 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 33745 akzeptiert
<--- 350 Restarting at 196608. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 245760
---> PORT 62,171,163,52,154,59
---> REST 245760
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 245760 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 33743 akzeptiert
<--- 350 Restarting at 245760. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 262144
---> PORT 62,171,163,52,147,221
---> REST 262144
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 262144 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 38237 akzeptiert
<--- 350 Restarting at 262144. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket
---- Verbinde mit my dyndns.de (xx.xxx.xx.xxx) Port 669
---> FEAT
<--- 220 FRITZ!Box7560(UI) FTP server ready.
<--- 211- Extensions supported:
<---  UTF8
<---  MDTM
<---  SIZE
<---  AUTH TLS
<---  PBSZ
<---  PROT
<--- 211 end
---> AUTH TLS
<--- 234 Authentication method accepted
---> OPTS UTF8 ON
---> USER ftpuser
Certificate verification: subjectAltName: »my dyndns.de« matched
<--- 530 Please login with USER and PASS.
<--- 331 Password required for ftpuser.
---> PASS XXXX
---> OPTS UTF8 ON
---> PBSZ 0
---> PROT P
<--- 230 User ftpuser logged in.
<--- 200 UTF8 ON
<--- 200 ok
<--- 200 Data channel will be secured
---> TYPE I
---> SIZE config_files_2021-09-22.tar.gz
<--- 200 Type set to I.
<--- 213 294912
---> PORT 62,171,163,52,195,247
---> REST 294912
---> STOR config_files_2021-09-22.tar.gz
copy: put kehrte zu 294912 zurück, suche entsprechendes get
<--- 200 PORT command sucessful.
---- Datenverbindung von (xx.xxx.xx.xxx) Port 35583 akzeptiert
<--- 350 Restarting at 294912. Send STORE or RETRIEVE to initiate transfer.
<--- 150 Opening BINARY mode data connection for 'config_files_2021-09-22.tar.gz'.
Certificate verification: subjectAltName: »my dyndns.de« matched
---- Schließe den Daten Socket
<--- 426 Data Connection: Connection reset by peer.
---> QUIT
---- Schließe den Kontroll - Socket

With lftp 4.8.4 i got gnutls_record_recv: Die TLS-Verbindung wurde nicht ordentlich beendet. Assuming EOF.

Nov 13 '21 19:11 tome9111991

I came here because of this problem, too. I don't use SSL/TLS.

=> Oh, sorry. It works, but it takes a long time.

Jan 20 '22 06:01 gnh1201

when i experienced this, it turned out to be a false negative because of how i was using the command. in my case, i told it a file existed (when it didn't) for a parallelized mput command.

i setup a script to transfer a, b, and c from my local computer to example. turns out b wasn't there. lftp notifies me on launch that it couldn't find b, but then continued to upload a and c. once it was done, it stated waiting for transfer to complete at 100% for awhile and then sent a non-zero error code. i checked example and sure enough, a and c were complete.

my solution was to find the circumstances when b wasn't present and modified the script. no error now.

May 19 '22 21:05 MJGTwo

+1 lftp version 4.9.2

Does not work:

lftp -c " \
    set ftp:ssl-allow true; \
    set ftp:ssl-force true; \
    set ftp:ssl-protect-data true; \
    open -u <username> -e \" \
        mirror -X .* -X .*/ --reverse --verbose --delete source_dir/ target_dir/; \
        bye\" \
    remote_host \
"

Does work:

lftp -c " \
    set ftp:ssl-allow false; \
    open -u <username> -e \" \
        mirror -X .* -X .*/ --reverse --verbose --delete source_dir/ target_dir/; \
        bye\" \
    remote_host \
"

Although this gives me a workaround, I do not consider it a solution in any sense -.-

Feb 17 '23 13:02 simon-amadeus

lftp lftp copied to clipboard

Upload hang at 100%

lftp
lftp copied to clipboard