lapis icon indicating copy to clipboard operation
lapis copied to clipboard

Download large files from MySQL

Open edubart opened this issue 5 years ago • 1 comments

I am working on website where I want to download large files from MySQL, but I get this error when selecting:

Internal server error:
.../share/lua/5.1/lapis/db/mysql.lua:137: packet size too big: 5957133
stack traceback:
	[C]: in function 'assert'
	.../share/lua/5.1/lapis/db/mysql.lua:137: in function 'select'
	.../share/lua/5.1/lapis/db/base_model.lua:562: in function 'find'

To fix lua-resty-mysql needs to have this option "max_packet_size = 16777216" however setting in the lapis config does not work, I've checked the lapis mysql driver code, it ignores this option or any other custom lua-resty-mysql option.

Workaround:

Before doing the query, make sure that there is a db connection do:

ngx.ctx.resty_mysql_db._max_packet_size = 16777216

edubart avatar Aug 07 '18 18:08 edubart

the mysql config block is currently used to extract options to create the connection. This means that any custom connection parameters must be listed in our codebase to be copied to be passed through. Although I can add max_packet_size, this doesn't handle the other options, and won't work for the future. I'm thinking of adding some kind of resty_mysql_connect_options field that will be merged into the connect options unchanged.

Alternatively, the entire mysql config can be passed into resty mysql connect, but that makes it harder to initialize connections if you switch the drivers, since you may pass options that aren't supported or are incorrectly formatted.

leafo avatar Aug 07 '18 23:08 leafo