node-mysql2 icon indicating copy to clipboard operation
node-mysql2 copied to clipboard

Faster result processing using USE RESULT

Open infusion opened this issue 10 months ago • 2 comments

Hi,

thank you very much for the mysql2 library! I have a question that if not currently possible could be turned into a feature request. I want to process large streams of data from MySQL. The MySQL CLI client allows to add the -q option for avoiding buffering and PHP for example allows to add the result mode MYSQLI_USE_RESULT to stream the data. I only found a way to stream data using mysql2, but what I'm looking for is a way to run a query without buffering, optimally like so:

db = await mysql.createConnection({...})

for await (let row of db.query("SELECT * FROM t1")) {...} 

Is this already possible or is something in this direction already planned?

Thanks! Robert

infusion avatar Feb 03 '25 10:02 infusion

you can read results without buffering right now like this:

    conn.query("SELECT * FROM table")
            .on('result', function (row) {
                 // use the data
            })

I do want to add async iterator api for the same functionality, see discussion here for example: https://github.com/sidorares/node-mysql2/pull/822#issuecomment-409415308

there are more issues discussing similar topic, feel free to link them @infusion if you find something that gives a better context

sidorares avatar Feb 03 '25 22:02 sidorares

Hi

I have huge amount of data like 2 lakhs of returned by a query. And it is wrking fine amd taking less time in every db querying tool but using mysql2 node driver it is taking more time to buffer and return the data that can be stored in js object. Can you help me here how to solve this.

The above streaming thing is not working with pool connection. It says .on is not a function

nikhileshSakalabhaktula avatar Mar 08 '25 05:03 nikhileshSakalabhaktula