Add volume group handling

[DK101010]

Description

Motivation: Some applications needs an specific volume controller configuration. For example to get a optimal performance or it is necessary to get an certificate. Therefore admins needs a opportunity to add an volume to certain controller.

Implementation: Decided to add a VolumeGroup class as an abstract object to manage this feature. A volume group is a triple of vm id, volume id and group number.

• group number == bus number
• groups are optional, user can work like before
• groups can only created via api
• during the attach the group will be create
• if vm have got mixed volumes with and without groups, attach order is: root vol, vol with groups(order by disk seq), vols without groups(order by disk seq).
• for import unmanaged VMs user have got three option: 1. can do like before (no groups) => import with current conf, after restart volumes will be attached on the first controller until max and then second controller and so on. (applied after restart) 2. add vol groups to disk offering list => add groups like user want it (applied after restart) 3. set useControllerConfiguration = true => read current controller conf of unmanaged vm and add automatic volumegroups for each volume

SQL for Update:

DROP TABLE IF EXISTS `cloud`.`volume_group`;
CREATE TABLE IF NOT EXISTS `cloud`.`volume_group`(
    `id` bigint unsigned PRIMARY KEY AUTO_INCREMENT COMMENT 'Primary Key',
    `vm_id` bigint unsigned NOT NULL COMMENT 'foreign key to vm table',
    `volume_id` bigint unsigned NOT NULL COMMENT 'foreign key to volume table',
    `group_number` int unsigned NOT NULL COMMENT 'group number of volume',
    FOREIGN KEY(vm_id) REFERENCES vm_instance (id),
    FOREIGN KEY(volume_id) REFERENCES volumes (id)
);

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [x] Major
• [ ] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

[DK101010]

like the idea, is this (meant to be) vmware only?

In the future also for KVM but for now only for vmware.

[DK101010]

Hi @DaanHoogland, @nvazquez, I have a question, what I have to do to add follow code

def importUnmanagedInstance(self, command, method="GET"):
         response = importUnmanagedInstanceResponse()
         response = self.connection.marvinRequest(command, response_type=response, method=method)
         return response  

to

cloudstack/tools/marvin/marvin/cloudstack/cloudstackAPIClient.py

I added in my env, but it seems to be automated generated or ?

[DaanHoogland]

Hi @DaanHoogland, @nvazquez, I have a question, what I have to do to add follow code

cloudstack/tools/marvin/marvin/cloudstack/cloudstackAPIClient.py

I added in my env, but it seems to be automated generated or ?

yes, it should be auto generated.

[nvazquez]

Hi @DK101010 - just extending @DaanHoogland's answer. It should be generated by building marvin: mvn -P developer -pl :cloud-marvin and upgrade the packages pip install --upgrade tools/marvin/dist/Marvin-*.tar.gz

[DK101010]

Ahh, ok a case of self confusion :D. Thanks you two

[github-actions[bot]]

Hi @${author}, your pull request has merge conflicts. Can you fix the conflicts and sync your branch with the base branch?

[DK101010]

Soo, I guess I'm finished and ready for review :)

[DaanHoogland]

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

[DK101010]

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

[DaanHoogland]

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

:grin: than do so ;)

[DK101010]

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

😁 than do so ;)

Oh, how I can do it? :D I can only editing the description and the headline.

[DaanHoogland]

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

grin than do so ;)

Oh, how I can do it? :D I can only editing the description and the headline.

You don't see the Ready for review button below here?

[DaanHoogland]

also @DK101010 , if you go to files you should be able to commit multiple suggestions in one commit.

[DK101010]

also @DK101010 , if you go to files you should be able to commit multiple suggestions in one commit.

Thanks for the hint, I will try it next time. Or I could squash the commits if there are too many.

[DaanHoogland]

@blueorangutan package

[nvazquez]

@blueorangutan package

[blueorangutan]

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 3186

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[nvazquez]

Hi @DK101010 there are some conflicts on this PR, can you please fix?

[blueorangutan]

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 3217

[acs-robot]

Found UI changes, kicking a new UI QA build @blueorangutan ui

[blueorangutan]

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

[blueorangutan]

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6012 (SL-JID-1454)

[acs-robot]

Found UI changes, kicking a new UI QA build @blueorangutan ui

[blueorangutan]

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

[blueorangutan]

UI build: :heavy_check_mark: Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6012 (SL-JID-1457)

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@blueorangutan package