Issue during upload of backup in external location

[baldodavi]

Hi all. I experiment a strange behavior during backup restorepoints creation. Upload in external location seems have some issues. In next image you can seethe whole data files created in Local storage backup after "Create full restoring point".

Here you can see same folder in another nextcloud using as external location.

The last file is not uploaded and the third one is created but upload seems fails. Same behavior using as external drive an Amazon S3 drive.

I test this feature using docker installation and 1.0.4 version of Backup app.

[mastermns]

Hey,

can you try to run the upload process manually ? Maybe it runs into a Problem.

"occ backup:point:upload -v --external "

[baldodavi]

So sorry but i'm not a nextcloud expert user. Executing cloud@ns3100443:/opt/cloud/next-cloud-test$ docker exec -it -u 33 next-cloud-test_app_1 /var/www/html/occ backup:point:upload -v --external

obtain

[Symfony\Component\Console\Exception\RuntimeException] The "--external" option requires a value.

I tried to specify external storage name but I have this result: [Symfony\Component\Console\Exception\RuntimeException] Not enough arguments (missing: "point").

What have I miss? The real name of external storage?

[mastermns]

No problem.

First run the command:
docker exec -it -u 33 next-cloud-test_app_1 /var/www/html/occ backup:point:list

You should not get an overview of your storage destinations as well as the storage point ids

• retreiving data from local

found RestoringPoint 20220103211601-full-xwdqwe12eaeeasda

• retreiving data from external:30

found RestoringPoint 20220103211601-full-xwdqwe12eaeeasda

Once you have this overview you can run the initial command: (here with my example) docker exec -it -u 33 next-cloud-test_app_1 /var/www/html/occ backup:point:upload -v --external external:30 20220103211601-full-xwdqwe12eaeeasda

[baldodavi]

I've to different nextcloud instances. The first have finish upload of packed backup the next day after 24 hours.

The second seems have some issues during packing. Is there a command permit to test packing?

docker exec -it -u 33 test-cloud2_app_1 /var/www/html/occ backup:point:list
- retreiving data from local
 > found RestoringPoint 20220105160002-full-Pibqbw2MfC6mCSa
 > found RestoringPoint 20220105221503-full-KQGupM5Ar61vrQw
 > found RestoringPoint 20220108222002-differential-j8i9kJk2bSYAEJP
- retreiving data from external:3
 > found RestoringPoint 20220105160002-full-Pibqbw2MfC6mCSa
+----------------------------------------------+---------------------+-----------+----------+-------------------------------------+---------+------------+------------+------------------+--+
| Restoring Point                              | Date                | NC        | Database | Parent                              | Comment | Status     | Instance   | Health           |  |
+----------------------------------------------+---------------------+-----------+----------+-------------------------------------+---------+------------+------------+------------------+--+
|  20220105160002-full-Pibqbw2MfC6mCSa         | 2022-01-05 16:00:02 | 23.0.0.10 | mysql    |                                     |         | packed     | local      | 5D, 23H, 39M ago |  |
|                                              |                     |           |          |                                     |         | packed     | external:3 | not complete     |  |
|  20220105221503-full-KQGupM5Ar61vrQw         | 2022-01-05 22:15:03 | 23.0.0.10 | mysql    |                                     |         | not packed | local      | not checked      |  |
|  20220108222002-differential-j8i9kJk2bSYAEJP | 2022-01-08 22:20:02 | 23.0.0.10 | mysql    | 20220105221503-full-KQGupM5Ar61vrQw |         | not packed | local      | not checked      |  |
+----------------------------------------------+---------------------+-----------+----------+-------------------------------------+---------+------------+------------+------------------+--+

[mastermns]

actually there is

occ backup:point:pack -v <POINTID>

once it is packed you can upload it

[baldodavi]

I'm going to make another full restoring point backup to monitoring timing. The general behavior, whit my installation method by the way, of backup tool is not very responsive: 1 -backup starting (pressing on Create full request restoring point) take some time to start. I'm trying to check this time 10 minutes ago and the red sentence "The creation of a restoring point as been requested and will be initiated soon." is still there and backup not started. Last time I've monitor this task, it started after several hours. 2- operations of packing and uploading of backup take some time to start after finish of previously operation. Sometimes fail, as shown, but seems there's some retry scheduled in 24 hours.

Is this behaviour normal or there's something wrong in my installation?

I'm going to report what happaning with new full restoring point creation.

[baldodavi]

Here the error, any advise?

docker exec -it -u 33 test-cloud2_app_1 /var/www/html/occ backup:point:upload -v --external external:3 20220113094501-full-tr46zSjngCrODFS
- uploading 20220113094501-full-tr46zSjngCrODFS to external folders
 - checking external folder 3:/restorepoints
  > restoring point not found
  * Creating Restoring Point on external folder: ok
  > restoring point created
  * initiating health check
  > Health status: 0 correct, 167 missing and 0 faulty files
  * Uploading data/data-ae49bc1c-727d-4594-afc8-ee3ac9184efa/00001-HsO4OIesi6Wc5BR: ok
  * Uploading data/data-ae49bc1c-727d-4594-afc8-ee3ac9184efa/00002-1QXBEjHMlZtW1uo: PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 104857920 bytes) in /var/www/html/lib/private/Files/Storage/Common.php on line 208
PHP Fatal error:  Uncaught Error: Class "OC\Log\ExceptionSerializer" not found in /var/www/html/lib/private/Log.php:316
Stack trace:
#0 /var/www/html/lib/private/Log/ErrorHandler.php(93): OC\Log->logException(Object(Error), Array)
#1 [internal function]: OC\Log\ErrorHandler::onError(2, 'fopen(/tmp/oc_t...', '/var/www/html/a...', 641)
#2 /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php(641): fopen('/tmp/oc_tmp_iV7...', 'r')
#3 /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php(501): OCA\Files_External\Lib\Storage\AmazonS3->writeBack('/tmp/oc_tmp_iV7...', 'restorepoints/2...')
#4 [internal function]: OCA\Files_External\Lib\Storage\AmazonS3->OCA\Files_External\Lib\Storage\{closure}()
#5 /var/www/html/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php(119): call_user_func(Object(Closure))
#6 [internal function]: Icewind\Streams\CallbackWrapper->stream_close()
#7 {main}
  thrown in /var/www/html/lib/private/Log.php on line 316

Is related with https://github.com/nextcloud/backup/issues/136 ?