sqlite-vss icon indicating copy to clipboard operation
sqlite-vss copied to clipboard

JSON / buffer performance

Open tap2k opened this issue 1 year ago • 1 comments

Are there any performance differences between using buffers and JSON strings to store embeddings?

FYI on node I am only able to get node-sqlite3 (and not better-sqlite) to work with JSON storage.

tap2k avatar Dec 12 '23 13:12 tap2k

Whether you insert vectors as buffers or JSON, it'll be stored in the same way, so there's no storage-related benefit either way. There may be some small CPU perf improvement if you use buffers instead (since it doesn't need to serialize/deserialize JSON), but you probably won't notice much of a difference.

Insert data with JSON in better-sqlite should work the same - do you mind sharing some sample code? Be sure to call JSON.stringify() before passing in vectors

asg017 avatar Dec 12 '23 17:12 asg017