Feature request: FreeCapacity and TotalCapacity for resource groups

[kvaps]

Hi there is need an opportunity to show query FreeCapacity and TotalCapacity per resource group/

This is information is really useful to collect usage statistics by linstor drivers (eg. for OpenNebula)

[rck]

I love that. It will make it so much easier for me to finger point at LINSTOR if users complain about how plugins show that information :+1: :smile:

"But thinly provisioned", "but my local node only", "but the whole cluster", "but I want it taking redundancy into account", "but no, not redundancy, that does not make sense at all!1 overall is the only way!",...

[ghernadi]

Not sure if that is what you are talking about, but what about the query-max-volume-size feature?

LINSTOR ==> rg qmvs rg1
╭─────────────────────────────────────────────────────────────╮
┊ StoragePool ┊ MaxVolumeSize ┊ Provisioning ┊ Nodes          ┊
╞═════════════════════════════════════════════════════════════╡
┊ lvmpool     ┊      9.89 GiB ┊ Thick        ┊ bravo, charlie ┊
╰─────────────────────────────────────────────────────────────╯

[rck]

If "you" is me, then no, this is only part of the story

[raltnoeder]

Resource groups are not required to be associated with a storage pool, so they cannot report statistics about the free or total capacity of "the resource group". A resource group is really just a profile, a set of properties, to apply to resources.

[rck]

it could give an answer to free/total given that one assumes that it is an "autoplace". Then the autoplace yields a concrete answer like "node1:poolA", "node2:poolB". Given that, one could invent something what that means for free/total. Like (and not saying it is useful in any way) summing up the indidual free/totals for the nodes and chosen storage pools. The question is how such a function should look like to be meaningful. So the question is always "What are free/total if I run autoplace on the given RG with the given autoplace constraints?".

[raltnoeder]

That's the problem, you would probably get different data every time you run it, not useful for anything.

[rck]

That is not true, there are use cases where it makes sense. For example "DRBD storage" in Proxmox is configured via a RG for every "storage" you define. Its properties are very much static. The API also requires a "give me free/total space" callback. So here it would make sense to just get that information out of the RG without doing some manual calculations in the plugin itself (like checking which SP is used for the RG and then iterating+summing all nodes that have that SP). So there are use cases. And getting free/total even makes sense if the properties of a RG change. What it assumes is that "autoplace" is stable.

[kvaps]

That's the problem, you would probably get different data every time you run it, not useful for anything.

That's not a problem at all, ZFS already doing that for many years.

[ghernadi]

Correct me if I am wrong, but to my understanding I think the problem here is the following:

From linstor's perspective, it's really hard / not obvious to tell anything about "the affected" storage pools, as an autoplace or spawn could return different results when called multiple times (free space of i.e. thin pools might even change without having new resource created, node-properties might have been added between two calles, etc...).

From a plugin's perspective, some basically say "I know that this resource-group will only operate on this (set?) of storage pool(s) - now I'd like to have statistics/metrics" - which also makes sense.

However, these two perspectives are not directly compatible. The plugin has knowledge / assumptions that are not always correct in general, while linstor must also work for a general use-case. That means, in my opinion, we are missing a bridging component ("glue") here. But as rck already mentioned, what this glue-component should actually do (and also how) is mostly based on the user / the use-case.

So - in this case, as least I still have no clue how this glue-component should look like, and also if it should really be implemented in linstor or in the plugin..

[rck]

From linstor's perspective, it's really hard / not obvious to tell anything about "the affected" storage pools, as an autoplace or spawn could return different results when called multiple times (free space of i.e. thin pools might even change without having new resource created, node-properties might have been added between two calles, etc...).

I don't get this particular problem (if we assume the algorithm is stable). Sure it will be different, sure it will change from call to call. That is what it is, a value that was true at the given time, which might be different in the next millisecond. "Welcome to storage"? If you ask your local file system how much is free and then write files and ask it again it also changes. Changes are not the problem here

[kvaps]

In my opinnion it might be done the same way like query-max-volume-size, eg:

# linstor resource-group query-usage-stats DfltRscGrp
╭─────────────────────────────────────────────────────────────╮
┊ StoragePool ┊  FreeCapacity ┊ UsedCapacity ┊ TotalCapacity  ┊
╞═════════════════════════════════════════════════════════════╡
┊ lvmpool     ┊    354.89 GiB ┊   502.02 GiB ┊    856.91 GiB  ┊
╰─────────────────────────────────────────────────────────────╯

where:

• FreeCapacity - is calculated same way like query-max-volume-size by summing up all the FreeCapacity values for storagepools.
• UsedCapacity - is calculated by summing up all the (TotalCapacity - FreeCapacity) values for storagepools, or summing up all the volumes for resource-definitions in this resource group
• TotalCapacity - is (FreeCapacity + UsedCapacity).

[raltnoeder]

Simple scenario: I have a resource group profile that is configured to use storage pool main, and there are currently 3 resources associated with it, which all are in storage pool main. Now I query the statistics of the resource group for the first time. Then I change the resource group profile to use storage pool archive, and I spawn a new resource using the resource group. Now I have 3 resources in storage pool main and 1 resource in storage pool archive. I query the statistics of the resource group for the second time. Then I manually create a resource definition that I assign to the resource group profile, but I create the 6 resources manually and select the storage pool backup. Also, for some math fun, those resources are larger than the total capacity of storage pools main and archive combined. Then I query the statistics of the resource group for the third time.

What data do you expect each of the three queries of the resource group's statistics to produce? Does FreeCapacity report the currently selected storage pool? Does it matter that TotalCapacity - UsedCapacity does not equal FreeCapacity then? Or does it instead report the free space in all storage pools where it found at least one associated resource, so that TotalCapacity, UsedCapacity and FreeCapacity are consistent, but FreeCapacity is a useless value, because it does not actually use those other storage pools for creating resources? Or does it maybe report UsedCapacity and TotalCapacity only for the selected storage pool as well, but then you lose track of resources that were created before the storage pool for the resource group was changed?

And even if we choose one of those possibilities: How is this supposed to be at least somewhat intuitive to anyone who reviews the report data without knowing every detail of the cluster configuration?

[rck]

The only feedback I can (or am willing :D) to provide here:

• these wild changes in a RG are for sure something nice to think about, but from a practical point of view very unlikely. In Proxmox you would set the properties that matter (mainly SP and redundancy) once. In k8s you set them on a "storage class" which then truly is read only. So it is pretty hypothetical, but fair enough
• IMO one would always show the current state, no matter how crazily it might have changed.
• what's the function to then calculate a meaningful value? I don't know, otherwise I would have suggested one already a long time ago. I'm not even sure if such a metric exists. For Proxmox it looks like every user came up with a different function of "meaningful". Sum over SPs, sum of SPs / number of nodes,... but hey, that was my hope here when I said I take any metric and then I can just finger point :D. On the other hand there should be some prior art how this should look like.

Or we could play it the other way around: You have to write a plugin for your favorite virtualization platform. You chose LINSTOR RGs, because that is how to do it. The plugin API requires you to implement a free/total callback (actually that is exactly what Proxmox does). What do you do? Currently I (and all other plugins) have to come up with some metric. Guess you can understand that it would be nice to shift this manual "get the SP from the RG and then manually sum up stuff in the custer" to LINSTOR.

All over and tl;tr: it is a very valid request IMO, I'm sure it should reflect the current state (as in what would happen if I run autoplace now), I have no clue what the best way is to then sum up storage cluster wide. Let alone thin provisioning.