node
node copied to clipboard
feat(sqlite): add `StatementSync.prototype.iterate` method
Hello,
I wanted an iterate
method for StatementSync
. An SQL query can return lot of rows, for memory efficiency it seems important we can fetch rows on demand, instead collect all in an array.
I'm not a C/C++ developer, so this was really challenging to create an object with a next
callback with v8. I tried to obtain Iterator.prototype
so the results extends Iterator. I did not found how to do it with v8.
I'm sure my code is not ready to be merged:
- [x] memory leak.
- follow similar algorithm than
All
- ~~variables for
next
andreturn
callback are wrapped byv8::External
~~ added to iterable_iterator JS Object, accessed from context (.This()
)
- follow similar algorithm than
- [x] no unit tests
- I did not found unit tests for sqlite module, I can do some in JS, but in C++...
- Maybe they are not mandatory for experimental modules ?
- [x] solution to have an iterable iterator from
iterate()
is a bit hacky. in jssqlite
module I decorate the method to returnIterator.from(<result from cpp iterate>)
.- If someone can help me to extend Iterator from v8 it could be great, else I'll keep the decorator.
- [x] documentation
I'm hoping to get some help here to finalize this PR, so that we all get a quality iterate method.
Manual test:
// % ./node --experimental-sqlite
sqlite = require('node:sqlite');
db = new sqlite.DatabaseSync(':memory:');
stmt = db.prepare(`WITH cte(a) AS (VALUES (1), (2), (3)) SELECT * FROM cte`);
iterator = stmt.iterate();
// > Object [Iterator] {}
iterator.map(({a}) => a).map(x => x*x).toArray();
// > [ 1, 4, 9 ]
So it's properly integrated with JS iterator protocol, it's an iterable iterator, it's integrated with IteratoHelpers.