Do not enforce the default "--reserve-memory" if "--memory" was explicitly given

[nyh]

A user complained in https://groups.google.com/d/msg/scylladb-users/jWkYesNzz_g/Dcxkgt--CgAJ that he can't run a Seastar program with "--memory 512M" on a 2GB machine, getting the cryptic error message:

FATAL: Exception during startup, aborting: std::runtime_error (insufficient physical memory)

The problem is that core/resource.cc, calculate_memory(), enforces the default_reserve_memory defined there to be removed from the total available memory (there's an additional complication of 500MB but it doesn't change the outcome so let's ignore it for now) - not leaving enough memory for the --memory option.

I think this logic is wrong. The way users would probably view the two options "--memory" and and "--reserve-memory" (and also how I explained it in the Seastar tutorial...) are as two alternative ways to specify the amount of memory given to Seastar - either by specifying the amount to use, or specifying the amount not to use (and Seastar should use the rest). If the user specifies just one of these options, he should be responsible for the implications of the amount he chose (he must leave what he considers to be enough memory for both Seastar and the OS) and he shouldn't be bothered with the default value of the other option.

[raphaelsc]

I think it's better for us to decide this than the regular user. Advanced users which are more aware of OS internals (how much memory per CPU it needs?) is left with the choice to override reserved memory. That allows us to improve the heuristic over time and provide user with a better experience. I think --reserve-memory is indeed confusing, and perhaps should be renamed to --override-reserved-memory, and --reserve-memory would be deprecated but kept for backward compatibility.

[nyh]

@raphaelsc I don't agree. I think that out-of-the-box users will not touch either options, but if a user chose to use "--memory" (-m), he clearly meant this command to be obeyed, and not ignored because Seastar thinks it knows better. Especially when it doesn't, really, and this 1.5 GB number is a completely made up number (it used to be 0.5 GB, then 2 GB, and today 1.5 GB, and none of these changes made any real sense except of fitting someone's usecase-du-jour).

In any case, both Seastar tutorial and the http://docs.scylladb.com/kb/memory-usage/ currently describe the two options, --memory and --reserve-memory, as two alternatives which you should choose at your convenience. It doesn't say anything about the default reserve (which the second document doesn't even mention!) being applied even when --memory is used, which can cause boot to fail if not enough memory is available for both.

If we do keep the current behavior, like you are recommending, the least we can do is to change the error message, to say something about the sum of --memory (=...) and --reserve memory (=...) to be more than the total amount of memory (=...).

[travisdowns]

If the user specifies --memory and --reserve-memory at the same time, is the only use of --reserve-memory in that scenario to do the startup check & emit error if the sum is less the available memory or does it have another function?

I do agree that the name "reserve-memory" is also confusing: without reference to the documentation I would assume this is something like "reserve memory for seastar" rather than the opposite (though perhaps this is not very important with suitably clear and accessible documentation).