Add bytes and bits module

[MartinSStewart]

I wanted to represent bitrate, aka bits per second, when I noticed there isn't a module for bits. I guess there would be two units, Bits and Bytes, with helper functions to convert between them. The reason for this is that you'll almost always work with integer amounts of either one or the other.

The module might look something like this (plus wikipedia links to explain the difference between "kilo" and "kibi")

type Bits = Bits
type Bytes = Bytes

bytesToBits : Quantity number Bytes -> Quantity number Bits
bytesToBits (Quantity.Quantity bytes) = (Quantity bytes * 8)

bitsToBytes : Quantity Float Bits-> Quantity Float Bytes
bitsToBytes (Quantity.Quantity bits) = (Quantity bits / 8)

{-| https://en.wikipedia.org/wiki/Kilobit -}
kilobit : Quantity number Bits
kilobit = 1000

{-| https://en.wikipedia.org/wiki/Kibibit -}
kibibit : Quantity number Bits
kibibit = 1024

{-| https://en.wikipedia.org/wiki/Kilobyte -}
kilobyte : Quantity number Bytes
kilobyte = 1000

{-| https://en.wikipedia.org/wiki/Kibibyte -}
kibibyte : Quantity number Bytes
kibibyte = 1024

...

[ianmackenzie]

Intriguing! The bits/bytes thing is definitely a bit tricky, might ponder that for a while.

What would be the name of the module? Might be cleanest to have separate Bits and Bytes modules, then we could have Bits.asBytes and Bytes.asBits functions.

[MartinSStewart]

If it was two separate modules with Bits.asBytes and Bytes.asBits, won't you need to introduce an Internal module and place Bits and Bytes there in order to avoid a dependency loop? Not sure if that's an issue or not.

[ianmackenzie]

Hmm right, good point, I clearly didn't think that one all the way through!

Can you think of a name for a single module that would work and be unlikely to conflict with other modules? Something like BinarySize maybe?

[MartinSStewart]

I think just Bytes works. Both Bytes and Bits seem pretty discoverable in a Bytes module. BinarySize works too but all else being equal, Bytes is less to type.

[ianmackenzie]

Unfortunately Bytes kinda conflicts with https://package.elm-lang.org/packages/elm/bytes/latest/ 🙂

[ianmackenzie]

(which I should have thought of when I proposed it originally...)

[MartinSStewart]

Good point. Another option is Byte which to me sounds awkward as a module name, but has the advantage of being even shorter. Otherwise BinarySize still works.

Edit: Just to be sure I checked and neither module causes a naming conflict https://klaftertief.github.io/elm-search/?q=module%3AByte https://klaftertief.github.io/elm-search/?q=module%3ABinarySize

[ianmackenzie]

It occurs to me we might be able to be 'cheat' a bit by exploiting the fact that 1000 is evenly divisible by 8, which means that an integer number of kilobits (or megabits, gigabits etc.) is in fact an integer number of bytes. So we could just use Bytes as the unit type and have

kilobits : number -> Quantity number Bytes
kilobits numKilobits =
    bytes (125 * numKilobits)

kibibits : number -> Quantity number Bytes
kibibits numKibibits =
    bytes (128 * numKibibits)

In that case the only big question might be what the signature of bits should be:

-- Kind of weird but I'm guessing 'number of bits'
-- will usually be an integer
bits : Int -> Quantity Float Bytes

-- A bit more normal but might require some
-- pointless-seeming 'toFloat' calls
bits : Float -> Quantity Float Bytes

-- Not implementable as far as I can see
bits : number -> Quantity Float Bytes

[MartinSStewart]

The issue with not having a Bits type is that I can't represent an integer number of bits. I want to decode an http request from Discord which contains an int representing bits per second. I bet there are other use cases (probably involving networking or file decoding) where you might be working with an integer quantity of bits that doesn't divide by 8.

[ianmackenzie]

Hmm yeah, I guess with binary file/network formats it would be pretty common to need to offset things by an integer numbers of bits. Back to the drawing board...

[ianmackenzie]

One more thought: even with the ability to specify an integer number of bits, something like

BinarySize.bits 12345 |> Quantity.per Duration.second

would still give you you a Quantity Float (Rate Bits Seconds). If you needed that rate to also be an integer quantity, we'd need a separate Bitrate or similar module with things like

type alias BitsPerSecond =
    Rate Bits Seconds

bitsPerSecond : number -> Quantity number BitsPerSecond

[MartinSStewart]

Currently I've been converting from raw ints to Quantity Int (Rate Bits Seconds) using JD.int |> JD.map Quantity since no scaling is needed. I suppose another module could be added for BitsPerSecond, but if you did that, would it also have a BytesPerSecond then?

[MartinSStewart]

Could Duration.seconds be changed to take number instead of Float since it doesn't apply any scaling? That would also make it easier to construct bitsPerSecond. The same could be done for any constructor that uses the base unit or can be scaled by an integer to exactly equal the base unit.

Obviously it would be a breaking change so it won't be something to do right this moment but maybe at some point in the future?

[ianmackenzie]

Could potentially do that but I don't think it would really help, since Quantity.per can only ever take and return Float-valued quantities anyways. It can't be changed to number because / only works on Floats, and besides you'd presumably still want

BinarySize.bits 5 |> Quantity.per (Duration.seconds 3)

to return a Float-valued quantity...

Philosophically, I'm also a bit reluctant to have things like Duration.seconds take a number instead of a Float since it makes the internal choice of units more meaningful and less of an implementation detail. To me it only really makes sense to support Int-valued quantities for things like bits and pixels which have a very clear fundamental discretization, which things like Length and Duration do not.

[ianmackenzie]

What about only having a Bits units type? Then you'd just have a Bits module something like:

module Bits exposing
    ( Bits
    , bits
    , inBits
    , bytes
    , inBytes
    , kilobits
    , inKilobits
    , ...
    )

type Bits
    = Bits

bits : number -> Quantity number Bits

inBits : Quantity number Bits -> number

bytes : number -> Quantity number Bits

inBytes : Quantity Float Bits -> Float

kilobits : number -> Quantity number Bits

inKilobits : Quantity Float Bits -> Float

-- etc.

I guess the main question is then whether there are cases where it's crucial to track with the type system that a particular Quantity represents an integer number of bytes, and be able to propagate that through operations like addition etc. But it seems to me that this design would cover a lot of use cases, perhaps with a few fairly clear and obvious uses of ceil and float such as

ceil (Bits.inBytes messageSize)

to get the size in bytes of a particular Protobuf message or something.

[MartinSStewart]

I guess the main question is then whether there are cases where it's crucial to track with the type system that a particular Quantity represents an integer number of bytes

Suppose someone is creating an AWS API and one of the endpoints requires the user to specify how large some resource should be. The API for it would look something like this createResource : Quantity Int Bytes -> OtherInfo -> Task Error (). If there was only Quantity Int Bits then the user could send in a nonsense size like 100.125 bytes. The API author would probably settle for using Int or defining their own Bytes type instead which kind of undermines having an official Bits module.

[MartinSStewart]

I guess maybe there isn't a good solution and it's better to not have an official Bytes/Bits module. Instead people will need to make a binary unit that suits their use case

[ianmackenzie]

I agree, I think there probably is enough trickiness here that I'd be a bit hesitant to immediately add an 'official' module to elm-units. But there are a lot of different alternatives to an official module:

• people defining their own binary units type in their apps, as you suggest
• a separate elm-binary-size or similar package based on elm-units that could be a bit more experimental and go through a few major versions before being merged back into elm-units
• an elm-units-extra package (I could even maintain that myself...) that could contain some 'experimental' elm-units modules (for potential eventual inclusion in elm-units) but evolve more rapidly/not have the same stability guarantees as elm-units

Thoughts?

[MartinSStewart]

a separate elm-binary-size or similar package based on elm-units that could be a bit more experimental and go through a few major versions before being merged back into elm-units

I think a package for only this is too small. The risk I see is that people won't bother installing it because it's quick to implement themselves so they won't think to look for it, or won't want to need to install yet another dependency.

an elm-units-extra package (I could even maintain that myself...) that could contain some 'experimental' elm-units modules (for potential eventual inclusion in elm-units) but evolve more rapidly/not have the same stability guarantees as elm-units

This could be a way to go. Especially if there's other stuff in it such as text formatting for units. That said, what does the process look like for moving module from elm-units-extra to elm-units? I guess the module has to be removed from elm-units-extra at the same time it's added to elm-units? Otherwise there will be a name collision.

[ianmackenzie]

Good point! I think the best approach would be to prefix the modules like Units.Extra.BinarySize, then rename to BinarySize when they get merged into elm-units...would be consistent with the elm-units-prefixed package I have to publish at some point (https://discourse.elm-lang.org/t/project-idea-prefixed-package-generator/5297).