units module_utils parser/formatter

[russoz]

Summary

Many modules need to either parse or format numbers in specific units, like kb, mb, gb, etc, or sometimes to format numbers as such. As of now, there seems to be little code reuse and therefore consistency in that front. Each module implements that logic in its own way. So the motivation for writing one such util is to achieve that consistency, reuse, and make modules simpler.

Thing to keep in mind/Features wanted/Comments in general:

• Many modules call external actors, such as CLI commands or APIs, and the need to either pass or parse that kind of information, so the parser/formatter should be flexible to allow some degree of customization, so that, despite using the same object, each module can meet the requirements of the external actors it interacts with
• In many occasions, there are subtle differences for the "same" (or close enough) cases MB, mB, MiB, ...
• It could/should have a mechanism to deal generically with different units (if it is a storage module it will use MB, GB, TB, but a network one will use Mbps, Gbps, Tbps). The generic mechanism should allow customization of specific cases.
  • This mechanism should be easily extensible - adding new units should be easy
• Another customization required is the minimum and maximum magnitude representations: some commands or APIs will not understand a higher (or lower) unit. Say, in this one particular module, we can talk about TB or PB, but EB is too much.
• Eventually, that module util could be encapsulated in filters (yup, some overlapping with ansible.builtin.human_to_bytes and its counterpart)

Issue Type

Feature Idea

Component Name

unit_format

Additional Information

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description: None

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[felixfontein]

There's already generic code for that:

• ansible.module_utils.common.text.formatters.human_to_bytes converts strings to numbers
• ansible.module_utils.common.text.formatters.bytes_to_human converts numbers to strings

There are also made available as module.human_to_bytes and module.bytes_to_human.

Unfortunately units are complicated, and while a lot of folks and tools use MB for mebibytes (2**20 bytes), others use that for megabytes (10**6 bytes) and expect MiB instead. The tools above fall in the first category.

Especially with modules for tools that use a different terminology this is pretty fatal, and I guess one of the reasons why this has been reimplemented some times.

[russoz]

The concerns you mentioned are pretty much the problem I was thinking of solving. I think those different behaviors can be parametrized.

[rstory]

I'd like to throw in a request for something to be able to simply get the K/M/G/etc prefix for things other than bits/bytes. ie with a specified string for units. eg. pps (packets/sec), qps (queries/sec), etc...

[russoz]

@rstory that's exactly one of the ideas - I may need to improve my writing, but that was what I was trying to achieve with the third bullet point in the summary.