Uploading large file as base64 string to parse server throws an error

[samateja]

New Issue Checklist

• [x] I am not disclosing a vulnerability.
• [x] I am not just asking a question.
• [x] I have searched through existing issues.
• [x] I can reproduce the issue with the latest version of Parse Server.

Issue Description

Uploading an recorded video of 17mb from ionic android to parse server throws an error.

Steps to reproduce

• Ionic media capture record 5 or 10 or 30 seconds video
• convert to base64 - which gives 17mb of string
• create parse.save('filename', {base64: base64data});

-- Which throws an error at RegExp.test().

Looking for help.

Actual Outcome

Expecting to upload video same as image

Expected Outcome

Expecting to upload video same as image

Environment

version --- "^3.4.3",

Server

• Parse Server version: "^5.0.0-beta.6",
• Operating system: Windows
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Local

Database

• System (MongoDB or Postgres): Mongo DB
• Database version: 1.31.3
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Local

Client

• SDK (iOS, Android, JavaScript, PHP, Unity, etc): Ionic Angular
• SDK version: android

[parse-github-assistant[bot]]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[mtrezza]

which gives 17mb of string

This seems to be an unusual approach to handle a file of that size. Storing 17 MB of string in a variable in memory is not the most robust concept. You may be hitting a memory limit with regex there. More investigation on your side would be needed to identify the root cause; find out where the error occurs - in the Parse client SDK or in Parse Server and at which line of code.

General note; please do not post the same question multiple times; you (and someone else with a different account) has been posting this issue in multiple places over the last 24 hours.

[FransGH]

Just stumbled over a similar issue in the JS SDK. Maybe this approach has now been added to other Parse SDK's as well. We suddenly got this error in one of our automated tests after years of not having any issue:

"Cannot create a Parse.File without valid data URIs or base64 encoded data."

As a fix, we modified some older code to pass a byte array instead (which makes more sense).

It took however some time to figure out what was happening here. Turns out Parse JS-SDK 3.3.4 over 3.3.0 has a (breaking?) change that is not immediately obvious until looking at the Parse.File source code.

In 3.3.4, this code is executed in Parse.File() to verify the base64 encoded file data:

        // Check if data URI or base64 string is valid
        const validationRegex = new RegExp(base64Regex.source + '|' + dataUriRegex.source, 'i');

And here the regex:

const base64Regex = new RegExp('([0-9a-zA-Z+/]{4})*(([0-9a-zA-Z+/]{2}==)|([0-9a-zA-Z+/]{3}=))', 'i');
const dataUriRegex = new RegExp(`^data:([a-zA-Z]+\\/[-a-zA-Z0-9+.]+(;[a-z-]+=[a-zA-Z0-9+.-]+)?)?(;base64)?,(${base64Regex.source})*$`, 'i');

With a ~2MB base64 string (jpeg image) it took around 6 minutes (!!!) with 100% cpu (=full single core) until the regex failed. The base64 was however correct as any image saved with Parse JS-SDK 3.3.0 and earlier could be downloaded/viewed without error.

IF there is a need to test for valid base64 data in Parse.File, a simple base64 decode would probably be more efficient (at the expense of using more memory).

But I would question that there is a need to test at all. Worse case the base64 data is invalid and either:

• save/upload fails (?)
• the storage adapter fails (?)
• a client reading it fails (most definately)

The 1st two can be caught by the client-app while uploading. The last one can also happen with pure binary data and is the client-app responsibility to catch, not the Parse SDK.

Just wanted to quickly comment on this so others might save some time troubleshooting. Problem is solved on my end and won't be using base64 upload anymore.

[mtrezza]

@FransGH Please take a look at https://github.com/parse-community/Parse-SDK-JS/issues/1510 as your issue may be a duplicate. If it's not could you please open a new issue to discuss this properly?

Closing this issue because of inactivity and no further feedback from OP.

[FransGH]

See #1532