Copy problem megatools

[darkistan]

When I start copying, I get the following error. The command is as follows megacopy --local /mnt/data/dump --remote /Root/SHADOW-SRV/HTZ error. ERROR: Upload failed for /Root/SHADOW-SRV/HTZ/vzdump-qemu-105-2021_09_28-23_15_1 8.vma.zst: API call 'p' failed: HTTP POST failed: CURL error: SSL connect error F

Tell me what is the reason for the error?

[darkistan]

[vukitoso]

I confirm there is an error

megatools 1.11.0

megatools copy --reload --no-progress --local=/tmp/site --remote=/Root/backup/day_2021-10-14_1634158800
ERROR: Upload failed for /Root/backup/day_2021-10-14_1634158800/files.zip: API call 'p' failed: HTTP POST failed: CURL error: SSL connect error

[vukitoso]

Solved the problem by rewriting the script code. I added multiple repetitions of the command through megatools. I try to loop through the command until it succeeds. My Bash function wrapper:

function run()
{
	echo "------------------------------------------------------------" | LOG
	OUTPUT=''
	OUTPUT_ERROR=''
		
	local cmd="$@"
	echo "CMD: [${cmd}]" | LOG
	
	local maxNumPut=10
	
	# Несколько попыток выполнения команды, при EXIT_CODE > 0
    for ((i=1; i <= $maxNumPut; i++))
    do
        echo "${FUNCNAME[0]}: $i попытка выполнения запроса" | LOG
        
        #${cmd} >${OUT} 2>${OUT_ERROR}
        eval "${cmd} >${OUT} 2>${OUT_ERROR}"
        EXIT_CODE="$?"
        echo "EXIT_CODE=[${EXIT_CODE}]" | LOG
        
        OUTPUT=`cat ${OUT}`
        echo "OUTPUT: [${OUTPUT}]" | LOG
        
        # Если файл с ошибками не пустой, то считываем его в OUTPUT_ERROR
        if [ -s "${OUT_ERROR}" ]; then
            OUTPUT_ERROR=`cat ${OUT_ERROR}`
            echo "OUTPUT_ERROR: [${OUTPUT_ERROR}]" | LOG
        fi
        
        # Успешная передача файла
        if [ "${EXIT_CODE}" == "0" ]; then
            return
        fi
        
        if [ $i == $maxNumPut ]; then
            echo "${FUNCNAME[0]} | ERROR | Команда выполнилась с EXIT_CODE > 0 за $maxNumPut попытки!" | LOG
            f_check_error
        fi
    done
}

run megatools put --no-progress --path="${DIR_REMOTE}" "'${FILE_LOCAL}'"

[grr]

it's not just limited to copying. when using megadl I will now routinely see: API call 'g' failed: HTTP POST failed: CURL error: SSL connect error. but the maintainer brushed this error off earlier in #268

[darkistan]

Решил проблему переписав код скрипта. Я добавил многократное повторение команды через мегаппараты. Я пытаюсь выполнить команду в цикле, пока она не завершится успешно.

function run()
{
	echo "------------------------------------------------------------" | LOG
	OUTPUT=''
	OUTPUT_ERROR=''
		
	local cmd="$@"
	echo "CMD: [${cmd}]" | LOG
	
	local maxNumPut=10
	
	# Несколько попыток выполнения команды, при EXIT_CODE > 0
    for ((i=1; i <= $maxNumPut; i++))
    do
        echo "${FUNCNAME[0]}: $i попытка выполнения запроса" | LOG
        
        #${cmd} >${OUT} 2>${OUT_ERROR}
        eval "${cmd} >${OUT} 2>${OUT_ERROR}"
        EXIT_CODE="$?"
        echo "EXIT_CODE=[${EXIT_CODE}]" | LOG
        
        OUTPUT=`cat ${OUT}`
        echo "OUTPUT: [${OUTPUT}]" | LOG
        
        # Если файл с ошибками не пустой, то считываем его в OUTPUT_ERROR
        if [ -s "${OUT_ERROR}" ]; then
            OUTPUT_ERROR=`cat ${OUT_ERROR}`
            echo "OUTPUT_ERROR: [${OUTPUT_ERROR}]" | LOG
        fi
        
        # Успешная передача файла
        if [ "${EXIT_CODE}" == "0" ]; then
            return
        fi
        
        if [ $i == $maxNumPut ]; then
            echo "${FUNCNAME[0]} | ERROR | Команда выполнилась с EXIT_CODE > 0 за $maxNumPut попытки!" | LOG
            f_check_error
        fi
    done
}

run megatools put --no-progress --path="${DIR_REMOTE}" "'${FILE_LOCAL}'"

А где именно лежит файл данного скрипта ?

[AssRap3r]

re-run your upload with --debug=api,http and look for the failure reason. this is what I got:

* Too old connection (1004 seconds), disconnect it
* Connection 0 seems to be dead!
* Closing connection 0
*   Trying 66.203.125.11:443...
* Connected to g.api.mega.co.nz (66.203.125.11) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL re-using session ID
* error:1421C0F8:SSL routines:set_client_ciphersuite:unknown cipher returned
* Closing connection 1

ERROR: Upload failed for 'file.bin': API call 'p' failed: HTTP POST failed: CURL error: SSL connect error

so it looks like it is a MEGA/OS issue and not specific to megatools?

debug on a successful upload:

*   Trying 66.203.125.12:443...
* Connected to g.api.mega.co.nz (66.203.125.12) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL re-using session ID
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=*.api.mega.co.nz
*  start date: Dec  6 00:00:00 2020 GMT
*  expire date: Dec  6 23:59:59 2021 GMT
*  subjectAltName: host "g.api.mega.co.nz" matched cert's "*.api.mega.co.nz"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
*        public key hash: sha256//0W38e765pAfPqS3DqSVOrPsC4MEOvRBaXQ7nY1AJ47E=
> POST /cs?id=1634916223&sid=KmBAVqNcLw_TmEecn12YfzNoMXVpU0lwZE40jPln6ZISXvlNz19UXv6Oxg HTTP/1.1

[vukitoso]

А где именно лежит файл данного скрипта ?

Это пример функции скрипта на bash. Я просто показал пример как можно решить данную проблему. Смысл в том, что я пытаюсь выполнить команду "megatools put ...." до тех пор, пока код возврата не будет нулём. Нолевой код возврата $? в bash означает успешное выполнение программы. Я написал на bash функцию обёртку для всех выполняемых команд через megatools. Даже "megatools ls ..." через обёртку запускаю. Помогло.

[Megous]

Maybe you can try disabling TLS session re-use in curl via:

curl_easy_setopt(curl, CURLOPT_SSL_SESSIONID_CACHE, 0L);

in http.c

[Megous]

I've added this change in the new builds. Maybe it will help.

https://megatools.megous.com/builds/experimental/

[vukitoso]

thx

[Megous]

I've re-built the package, because last one had wrong pcre build configuration that caused some issues.

[neodyne]

@megous both win32 and win64 builds of megatools-1.11.0-git-20211028 and megatools-1.11.0-git-20211029 fail to run with the following error:

[Megous]

@neodyne try the 20211030 one

[Megous]

Did the TLS session cache change help with uploads?

[AssRap3r]

@megous I've had no curl errors since updating to 20211030-linux-x86_64.tar.gz (8/8 success rate on uploads)

just re-tested the previous experimental version and got the SSL error on first upload. fixed for me, at least.

[Megous]

Great. Thanks for confirming. :)

[neodyne]

@megous FYI, from the build that fixed this (20211030) onwards, megatools (dl) cannot parse old-style mega (folder) links, for example this link (fails on both windows and linux, all builds):

./megatools dl 'https://mega.nz/#F!WwZyBZRL!1vXiBr7pJZLINpSRErBxvA' # 20211030, 20220401
ERROR: Can't open folder 'https://mega.nz/#F!WwZyBZRL!1vXiBr7pJZLINpSRErBxvA': Node not found:

Curiously, megatools built from corresponding source on ubuntu 20.04 doesn't exhibit this problem, so it seems this may be related to updated dependencies used in the published experimental builds.