speedb-io
db_bench: In multi-db runs, allow per-db options file
db_bench currently supports loading options from a file to open the DB-s participating in the run (via the -options_file flag). However, the same options file is used to configure all of the DB-s. This ticket is a proposal to allow the user to have a separate options file per DB.
@giladmaya - BTW, we will be happy to assist if you have any questions about this issue (and in general), or in case you would like to discuss, etc. Please feel free to contact us.
@udi-speedb - what do you think the syntax should be like in the case of multi-DB? would something similar to the "benchmarks" parameter fit? e.g. --options_file "path1,path2"
@giladmaya:
That makes sense. You may also look at --column_family_distribution or (our new) --dbs_to_use flag, coming in this release as a db_bench enhancement (https://github.com/speedb-io/speedb/issues/220)
A few issues that I think are worth considering:
- There might be multiple db-s, but you may wish to use a single options file. I think a user shouldn't always have to repeat the file's name to match the number of db-s.
- A convenience feature would be to allow specifying a base name (using another, optional flag) where all of the files would be located, and just specify the names of the individual files. That may be useful if all of your files are located under some folder, avoiding the need to repeat the folder's name again and again. However, that may be just a complication.
- We have (also in this coming release) added a benchmark groups mode. Please have a look at that as well. The db-s are opened only once, so the options_file flag would be applicable only for that group (https://github.com/speedb-io/speedb/issues/220)
What do you think?
@udi-speedb:
--column_family_distribution and --dbs_to_use are quite similar to my suggestion, will take a look at their implementation to see if there's anything I can reuse.
-
I think we should allow the user to either provide the same number of options files as the number of dbs or just a single options file (like the current behavior). This approach has another advantage since it's backward compatible.
-
I like the idea of specifying a directory instead of a folder, and it can definitely simplify things for users. However, I think that it should be implemented as another argument in the cli, e.g.
--options_dir. -
Will make sure my additions are supported in benchmark groups mode. Thanks for the reference.
@udi-speedb:
--column_family_distributionand--dbs_to_useare quite similar to my suggestion, will take a look at their implementation to see if there's anything I can reuse.
- I think we should allow the user to either provide the same number of options files as the number of dbs or just a single options file (like the current behavior). This approach has another advantage since it's backward compatible.
- I like the idea of specifying a directory instead of a folder, and it can definitely simplify things for users. However, I think that it should be implemented as another argument in the cli, e.g.
--options_dir.- Will make sure my additions are supported in benchmark groups mode. Thanks for the reference.
@giladmaya Sounds good to me