themis icon indicating copy to clipboard operation
themis copied to clipboard

File Encryption WASM Themis Memory Error abortOnCannotGrowMemory_emscripten_resize_heap

Open djaffer opened this issue 3 years ago • 11 comments

I am seeing this error when trying to encrypt a file of size ~20 mb. Tried to find example but couldn't find any.

image

djaffer avatar Jul 23 '22 02:07 djaffer

This is a critical breakage for us and please help to get it fixed ASAP. I have raised the PR above.

djaffer avatar Jul 23 '22 03:07 djaffer

Thank you @djaffer, we will take a look.

Meanwhile, please fill in the template so we can locate the issue easier:


To Reproduce Steps to reproduce the behavior:

  1. Use '...'
  2. Run '...'
  3. See the following error:
error

Expected behavior A clear and concise description of what you expected to happen.

Environment (please complete the following information):

  • OS: [OS name and version, e.g. iOS 13]
  • Hardware: [32-bit/64-bit, or mobile device name and version]
  • Themis version: [e.g. 0.12.0]
  • Installation way:
    • [ ] via package manager
    • [ ] built from source

Additional context Add any other relevant context for the problem here. Share an example project, if you can.

vixentael avatar Jul 24 '22 16:07 vixentael

does make LDFLAGS=-s ALLOW_MEMORY_GROWTH=1 help you? Or only if add this directly in the src/wrappers/themis/wasm/wasmthemis.mk file?

Lagovas avatar Jul 25 '22 22:07 Lagovas

You can test it once you add it and compile to ensure it is working. Maybe you have an alternative that works better.

djaffer avatar Jul 25 '22 23:07 djaffer

Most flags can be extended by explicitly specifying for make command and don't need to be hardcoded in the makefiles. Will be great if you try it like that. It will be much faster if you depend on Themis releases (not source code updates).

Lagovas avatar Jul 25 '22 23:07 Lagovas

It is a UI package, so I am not sure if it is the best course for us to compile. It is a genuine bug. Please resolve it whenever you can and use whatever way you think is best to fix the issue. For UI package the expectation should not be to compile for a basic feature to work.

djaffer avatar Jul 25 '22 23:07 djaffer

@djaffer

  1. please fill in the questionary so we know which Themis version you use, and how to reproduce the case. https://github.com/cossacklabs/themis/issues/928#issuecomment-1193355214

  2. What is the smallest file you can encrypt without issues?

  3. Please show the encryption code you use.

  4. Can you try using SecureCellContextImprint instead of SecureCellSeal to see if the issue persists? https://docs.cossacklabs.com/themis/languages/wasm/features/#context-imprint-mode

  5. if you can't test this flag yourself to make sure it really helps as @Lagovas suggested, you will need to wait until we can reproduce the case, test different fixes and release a fixed version. Instead of blindly adding the flag, we'd like to understand the root cause and make a conscious decision.

vixentael avatar Jul 26 '22 09:07 vixentael

@vixentael the fix was just a suggestion. I added the version details about when you first asked along with the code being used to encrypt how I am encrypting in the PR. Sure take your time and check validity. If you have themis on web playground I can try. I do not think it is a lot for your dev to just add an input file and test a 20 mb file.

I am closing this issue as I do not have more time to keep up with this. If you see it worthwhile you can fix it.

djaffer avatar Jul 26 '22 10:07 djaffer

now i see the updated PR. Thank you @djaffer, we will take a look

vixentael avatar Jul 26 '22 11:07 vixentael

@vixentael FYI same error with SecureCellContextImprint. Also tried in $(BIN_PATH)/libthemis.js: LDFLAGS += -s ALLOW_MEMORY_GROWTH=1

djaffer avatar Jul 26 '22 22:07 djaffer

This might also be related to https://github.com/cossacklabs/themis/issues/798

ilammy avatar Aug 03 '22 15:08 ilammy

i guess it's fixed in the latest release 0.14.7 https://github.com/cossacklabs/themis/releases/tag/0.14.7

@djaffer please take a look

vixentael avatar Aug 19 '22 14:08 vixentael

looks like the file size of libthemis.wasm has increased react build throwing error.

djaffer avatar Aug 21 '22 15:08 djaffer

The issue only happens in 0.14.7. The node module is 22 mb for this. The node_modules package for wasm has dist and dist-es6.

image

djaffer avatar Aug 21 '22 15:08 djaffer

hmm interesting side effects, @radetsky remember, we previously compiler Themis with O3 optimization flag?

vixentael avatar Aug 21 '22 19:08 vixentael

thank you for a feedback @djaffer!

we have released 0.14.8, that should be both: small & support large files. we have deprecated 0.14.7 as an unfortunate release. https://github.com/cossacklabs/themis/releases/tag/0.14.8

could you please take a look?

vixentael avatar Aug 23 '22 13:08 vixentael

It is working! Thank you! Size is now 10 MB. One more optimization you guys can do is to have a single copy libthemis.wasm rather than having it in each dist, dist6, and src directories. Removing the 2 extra copies could save 4 MB.

djaffer avatar Aug 24 '22 09:08 djaffer

thank you, @djaffer!

One more optimization you guys can do is to have a single copy libthemis.wasm rather than having it in each dist, dist6, and src directories. Removing the 2 extra copies could save 4 MB.

@radetsky that's a good one ^

vixentael avatar Aug 24 '22 12:08 vixentael