cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Quota] Update pre-populated Quota tariffs' type

Open BryanMLima opened this issue 1 year ago • 16 comments

Description

The PR #7152 changed the unit of the Quota types VM_DISK_IO_READ, VM_DISK_IO_WRITE, VM_DISK_BYTES_READ and VM_DISK_BYTES_WRITE. However, the pre-populated tariffs were not considered, and their unit type is still with the old value. The unit was only displayed incorrectly, as the calculation utilizes the correct unit. Their unit types were updated to the correct ones.

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] New feature (non-breaking change which adds functionality)
  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • [ ] BLOCKER
  • [ ] Critical
  • [ ] Major
  • [x] Minor
  • [ ] Trivial

How Has This Been Tested?

Update script 
2024-02-05 14:42:03,890 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) DB version = 4.19.0.0 Code Version = 4.19.1.0
2024-02-05 14:42:03,894 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Database upgrade must be performed from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:03,949 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Running upgrade Upgrade41900to41910 to upgrade from 4.19.0.0-4.19.1.0 to 4.19.1.0
2024-02-05 14:42:03,957 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Updates the populated Quota tariff's types VM_DISK_BYTES_READ, VM_DISK_BYTES_WRITE, VM_DISK_IO_READ and VM_DISK_IO_WRITE to the correct unit.
2024-02-05 14:42:03,966 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'Bytes', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_BYTES_READ', 'VM_DISK_BYTES_WRITE') 
2024-02-05 14:42:03,976 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'IOPS', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_IO_READ', 'VM_DISK_IO_WRITE') 
2024-02-05 14:42:04,091 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup upgrade Upgrade41900to41910 to upgrade from 4.19.0.0-4.19.1.0 to 4.19.1.0
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade cleanup from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:04,108 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,115 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup script Upgrade41900to41910 is executed successfully
2024-02-05 14:42:04,225 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Upgrade completed for version 4.19.1.0

After the update script, the usage units were updated accordingly.

|id|usage_type|usage_name|usage_unit|usage_discriminator|currency_value|effective_on|updated_on|updated_by|uuid|name|description|activation_rule|removed|end_date|
|--|----------|----------|----------|-------------------|--------------|------------|----------|----------|----|----|-----------|---------------|-------|--------|
|18|21|VM_DISK_IO_READ|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2dc7-771e-11ee-8e59-5254003754dc|VM_DISK_IO_READ|||||
|19|22|VM_DISK_IO_WRITE|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2ead-771e-11ee-8e59-5254003754dc|VM_DISK_IO_WRITE|||||
|20|23|VM_DISK_BYTES_READ|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2f84-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_READ|||||
|21|24|VM_DISK_BYTES_WRITE|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f304f-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_WRITE|||||

How did you try to break this feature and the system with this change?

BryanMLima avatar Feb 01 '24 13:02 BryanMLima

@JoaoJandre, do you think this can go to the 4.18.2.0 release, as it is a small fix?

BryanMLima avatar Feb 01 '24 13:02 BryanMLima

@blueorangutan package

BryanMLima avatar Feb 01 '24 13:02 BryanMLima

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Feb 01 '24 13:02 blueorangutan

Codecov Report

Attention: Patch coverage is 18.75000% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 31.49%. Comparing base (c795547) to head (3658e5b). Report is 47 commits behind head on 4.19.

Files Patch % Lines
...ava/com/cloud/upgrade/dao/Upgrade41900to41910.java 13.33% 13 Missing :warning:
Additional details and impacted files 
@@              Coverage Diff              @@
##               4.19    #8591       +/-   ##
=============================================
+ Coverage     20.99%   31.49%   +10.50%     
- Complexity    19683    35081    +15398     
=============================================
  Files          4849     5354      +505     
  Lines        324441   385758    +61317     
  Branches      45573    58895    +13322     
=============================================
+ Hits          68121   121507    +53386     
- Misses       246563   248242     +1679     
- Partials       9757    16009     +6252
Flag Coverage Δ
simulator-marvin-tests 25.46% <12.50%> (+4.47%) :arrow_up:
uitests 4.32% <ø> (?)
unit-tests 16.60% <18.75%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 01 '24 14:02 codecov[bot]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8489

blueorangutan avatar Feb 01 '24 14:02 blueorangutan

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

JoaoJandre avatar Feb 02 '24 16:02 JoaoJandre

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@JoaoJandre You need to add this to the upgrade schema of both 4.18.1-4.18.2 and 4.19.0-4.19.1

Considering this is a minor fix and this is the only database change between 4.18.1 and 4.18.2, can we retarget to 4.19.1 and add it to the upgrade schema of 4.19.0-4.19.1 only ?

weizhouapache avatar Feb 02 '24 17:02 weizhouapache

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@JoaoJandre You need to add this to the upgrade schema of both 4.18.1-4.18.2 and 4.19.0-4.19.1

Considering this is a minor fix and this is the only database change between 4.18.1 and 4.18.2, can we retarget to 4.19.1 and add it to the upgrade schema of 4.19.0-4.19.1 only ?

Retargeted to 4.19.1, as this is a minor fix.

BryanMLima avatar Feb 05 '24 14:02 BryanMLima

@blueorangutan package

BryanMLima avatar Feb 05 '24 14:02 BryanMLima

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Feb 05 '24 14:02 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8533

blueorangutan avatar Feb 05 '24 15:02 blueorangutan

@blueorangutan test

DaanHoogland avatar Feb 16 '24 13:02 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan avatar Feb 16 '24 13:02 blueorangutan

[SF] Trillian test result (tid-9257) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 56799 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9257-kvm-centos7.zip Smoke tests completed. 120 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_list_volumes_metrics_history Error 3607.52 test_metrics_api.py
test_02_trigger_shutdown Failure 336.64 test_safe_shutdown.py

blueorangutan avatar Feb 17 '24 05:02 blueorangutan

some CI failures, reopening.

DaanHoogland avatar Feb 20 '24 10:02 DaanHoogland

still doesn't pass :( though the error is in a library download. Applying insanity

DaanHoogland avatar Feb 20 '24 13:02 DaanHoogland

@blueorangutan package

DaanHoogland avatar Mar 04 '24 09:03 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 04 '24 09:03 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8844

blueorangutan avatar Mar 04 '24 10:03 blueorangutan

@blueorangutan test

DaanHoogland avatar Mar 04 '24 11:03 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan avatar Mar 04 '24 11:03 blueorangutan

[SF] Trillian test result (tid-9380) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 44248 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9380-kvm-centos7.zip Smoke tests completed. 129 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File

blueorangutan avatar Mar 05 '24 00:03 blueorangutan