sqlite_orm icon indicating copy to clipboard operation
sqlite_orm copied to clipboard

Published 20 hours ago • verified publisher icon fnc12

Crash when using a lot of tables

Open kelteseth opened this issue 2 years ago • 1 comments

I have a project with many tables, but every time I try to add the complete list of tables to my storage the app crashes. I created a python script that generates a complete test project with structs, sql and the make_table storage initialization (See Crash.zip). image

You can easily change the amount of generated tables in line 74 and 77. It always happens at about 40 (plus minus 10, I did not count) make_table.

Crash.zip

image

If you like you can use the python script for future testing and test data generation.

kelteseth avatar Sep 16 '22 12:09 kelteseth

Is there a way to work around this issue?

kelteseth avatar Sep 21 '22 11:09 kelteseth

what kind of crash you get? compiler crash or runtime crash?

fnc12 avatar Nov 02 '22 15:11 fnc12

@fnc12 this is a runtime crash. Tested with MSVC 2022 17.3.4. It persistently happens when using make_storage with a lot of tables. See my test project https://github.com/fnc12/sqlite_orm/files/9583738/Crash.zip

kelteseth avatar Nov 03 '22 10:11 kelteseth

what is error text?

fnc12 avatar Nov 03 '22 14:11 fnc12

@fnc12

It is a stack overflow, :) Is there any way to make make_storage allocate on the heap?

image

kelteseth avatar Nov 04 '22 10:11 kelteseth

you can try to allocate your storage on heap using new operator but it will not work cause stack is used for function call recursion in your case. All I can advice you is

  1. try the latest dev branch if you use any release instead
  2. try to split storage up to several storages. You can keep two or more storages with the same filename and they will work fine

fnc12 avatar Nov 04 '22 13:11 fnc12

@kelteseth It would be interesting whether you see an improvement with sqlite_orm v1.8.1.

trueqbit avatar Mar 11 '23 11:03 trueqbit

@kelteseth BTW thank you for providing the test case and the python script. It works for me - both at compile-time and run-time - using Visual C++ 17.5.3. This is expected because sqlite_orm is now using fold expressions for tuple iteration (i.e. loop unrolling) instead of recursive iteration.

trueqbit avatar Mar 24 '23 11:03 trueqbit

@trueqbit thank you🎉! I didn't have the time yet to test it, will report back soon hopefully 😊

kelteseth avatar Mar 25 '23 10:03 kelteseth

Can confirm it does work now in my project at work. Thanks, hopefully we will find the time to switch our db implementation and become a customer :3

kelteseth avatar Mar 29 '23 08:03 kelteseth