Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Improve inference export script

Open simonusher opened this issue 1 year ago • 1 comments

closes #3038

simonusher avatar May 07 '23 10:05 simonusher

I'm glad to hear that. I've addressed your comments except for one. Please let me know if the performance is sufficient. I believe the relationship filters, as they are now, will use correlated subqueries so they might be suboptimal. I'm not entirely sure how to test them properly at scale. Is there maybe a script that fills the development inference-db with dummy data?

simonusher avatar May 08 '23 17:05 simonusher

I'm glad to hear that. I've addressed your comments except for one. Please let me know if the performance is sufficient. I believe the relationship filters, as they are now, will use correlated subqueries so they might be suboptimal. I'm not entirely sure how to test them properly at scale. Is there maybe a script that fills the development inference-db with dummy data?

There isn't a way of filling the inference DB with dummy data currently, this would be a useful addition though so I'll raise a new issue for it. There's some dummy data here which was used for filling the data collection backend DB, could maybe be reused

olliestanley avatar May 08 '23 18:05 olliestanley

Thank you for this PR!

olliestanley avatar May 08 '23 18:05 olliestanley