flux-core
flux-core copied to clipboard
WIP content-sqlite: preallocate space to avoid outside diskfull events
Per discussion in #6169
Support a new "preallocate" config & module option to pre-allocate a specific amount of space to sqlite so that we can hopefully avoid ENOSPC issues when outside actors fill up the disk. Option name debatable ... "preallocate_size"?
The technique is to basically create a database table, write a bunch a data to it, then drop the table. This won't work with journaling b/c the journal needs space too. But code is added to disable journaling if ENOSPC is hit.
There's a lot of "setup" commits in here, we could probably split out into another PR.
TODOs
- [x] ~~option take meg/gig suffix?~~ ehhh scratch that, i don't what the config file inputs to always be strings
- [x] test at bigger scale/size (i.e. not 5m mounts)
- [x] I probably should document this somewhere
- [ ] BIG THING TO DISCUSS IN ISSUE - how to put database back into journaling mode when space issues fixed. Disabling journaling does lead to a healthy slowdown in performance (job throughupt maybe 15% down). This can probably be a different issue/PR, but needs to be discussed.