moosefs
[DEFAULT] Let's use `ATIME_MODE = 2` (relatime) by default
I have been using MooseFS (and LizardFS before it) for a while and I have years of operational experience of managing 150+ TiB cluster running variety of application and /home folders of several machines. As my partially SSD-backed MooseFS cluster was performing pretty well (with some quirks of SQLite3 as in Firefox), one application was running very slow with ~20 min. startup time and unresponsive GUI due to large data structures it was scanning and performing stat on every file. Just as I was thinking that I've exhausted all avenues of improving its performance, I've discovered that MooseFS master uses ATIME_MODE = 0 by default so it modifies access times excessively, causing significant performance degradation in some applications.
I was impressed how much performance and responsiveness of problematic application have improved after setting ATIME_MODE = 2 in /etc/mfs/mfsmaster.cfg.
Since ATIME_MODE = 2 is an equivalent of relatime mount option which is used by default by several file systems nowadays, I suggest to change default ATIME_MODE to 2.
I think we need a poll on that :) relatime is a really good invention, but is it the standard nowadays? For MooseFS itself it makes no difference which mode is on - if the change/update is needed, it's made in RAM anyway. But if there exist many applications that behave differently depending on the current mode (i.e. do NOT send a request to kernel with mode 2, that they would have sent with mode 0), then it makes a lot of sense.
I would love to see more comments from other MooseFS users on this particular subject.
Not an active MooseFS user for some time now (still waiting for that multi master version), but you asked for an opinion, so here it is.
Must admit that I'm 100% with @onlyjob on this one, for several reasons. Checking old mfs configurations I used several years ago, I see that setting ATIME_MODE to 4 (so, even more aggressively turning it off!) was the ONLY change from the default options I ever did. So, all defaults are sane, except that one. The reasons being:
- having it on, those master changelogs would really be written much more intensively than when turned off, it definitely does impact performance severely
- atime is one of those features that look great on paper, but NOBODY uses it in practice (name a single application that depends on it!)
- relatime really IS the current standard for fresh OS installations, and it has been like that for a decade, maybe even longer
tl;dr: yes, I think you should make ATIME_MODE = 2 the default as @onlyjob suggests, it's a good idea IMHO, and I think chances that anybody will complain about that change are very slim
Hope it helps!
I've used noatime on all my disks across all my systems for years now -- no real ill effects that I've noticed (even using Mutt, which is usually cited as a notable example that needs atime, although I only use it with IMAP so maybe I don't use it in a way that would be affected).
Similarly, I happily use ATIME_MODE=4 on my MooseFS deployments. :smile:
As much as I'd love to say that "4" is a sane default, I don't think it is generally, but I would definitely agree that "2" is a very reasonable one. :+1: :heart:
I vote for arime_mode=4 since moose master performance is critical.
We manage multiple PB of data.
I use atime_mode=4 on MooseFS, and have set noatime on all my systems for years now.
I say go for it. Set it to 2, or even 4.
@chogata - Most admins will likely know how paranoid they need to be tracking access times. With that said I would sway towards 4 on master + noatime for mfsmount by default as the most performant and IoT / consumer grade hardware friendly
Most admins will likely know how paranoid they need to be tracking access times.
Or not. I provide storage space to team/developers and I have little knowledge of how they are using it (or intend to use). (Though they will find me if/when something does not work...)
Even application requirements are usually not well documented.
But empirically everything works with ATIME_MODE = 2 as far as I can say.
