[Bug] Backup restore from Google Drive fails due to server error

[jnv]

1. Can you describe the problem and bug in more detail?

I can't fully restore my backup from Google Drive because every time it gets completely stuck at some point. Looking into the console, I can see this happens because Google's API eventually returns server error (500). Memex only prints this error into the console: Something went wrong making a request to Drive with Response object approximating to:

Response object

{
    "ok": false,
    "status": 500,
    "statusText": "Internal Server Error",
    "type": "basic",
    "url": "https://www.googleapis.com/drive/v3/files?q=%271Y6strOetDcz2ZYs9HK6Iv47ZOV1lChkIw_ZqMw6tPhdmlcA5ww%27%20in%20parents&pageToken=[REDACTED]&spaces=appDataFolder",
    "headers": {
        "alt-svc": "h3-29=\":443\"; ma=2592000,h3-27=\":443\"; ma=2592000,h3-T050=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
        "cache-control": "private, max-age=0",
        "content-encoding": "gzip",
        "content-length": "122",
        "content-security-policy": "frame-ancestors 'self'",
        "content-type": "application/json; charset=UTF-8",
        "date": "Wed, 12 Aug 2020 09:30:14 GMT",
        "expires": "Wed, 12 Aug 2020 09:30:14 GMT",
        "server": "GSE",
        "vary": "Origin, X-Origin",
        "x-content-type-options": "nosniff",
        "x-firefox-spdy": "h2",
        "x-frame-options": "SAMEORIGIN",
        "x-xss-protection": "1; mode=block"
    },
    "body": {
        "error": {
            "errors": [
                {
                    "domain": "global",
                    "reason": "backendError",
                    "message": "Backend Error"
                }
            ],
            "code": 500,
            "message": "Backend Error"
        }
    }
}

Screenshot

2. How can we replicate the issue?

• Go to Backup & Restore
• Select Restore & Replace
• Type "RESTORE"
• Select Google Drive
• Wait
• Open addon debugger to see the console with requests
• Wait some more
• The error eventually happens given sufficiently large backup, there's error in the console but no UI feedback.

3. Expected behavior (i.e. solution)

The restoration should be successful. If something goes wrong, I should at least get some feedback.

4. Error stack (from extension crash page)

See the screenshot in 1.

5. Other comments

Looking into the code, the error comes from the GoogleDriveClient:

https://github.com/WorldBrain/Memex/blob/93dcda9b161199d23feadf17e4ca3574fcaeba22/src/backup-restore/background/backend/google-drive/client.ts#L214-L224

There is no attempt to retry the request due to server error, it just crashes and spits out the error. Since server error from Google Drive is something we cannot control, I think it would be helpful to at least retry the request for a few times before completely failing.

I could take a stab at the retry part and prepare a PR. I see that there's retryUntil, so let's use it and retry the request under specific condition (status code must be 500), say, for 10 seconds with 1 second interval.

It would be still helpful to give a UI feedback to the user if something goes wrong (not to mention that clicking cancel doesn't seem to do anything).

[blackforestboi]

Does this error always happen at the same progression state of the import?

I searched around a bit but could not find a definite cause for this problem. Your solution may work just fine.

Thanks for the feedback and offer for a PR. Let me know If I can be of any help?

[jnv]

Does this error always happen at the same progression state of the import?

It doesn't, sometimes it happens around 10%, sometimes around 3%, the highest progress I got to was around 35%. I will take a look at it tonight, thanks!

[jnv]

By the way, I suspect that the user is not notified because of some bug in backup-restore/background, I am getting this error when fetch throws NetworkError:

It points to this line:

https://github.com/WorldBrain/Memex/blob/93dcda9b161199d23feadf17e4ca3574fcaeba22/src/backup-restore/background/index.ts#L176-L180