cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

Ability to specify NFS mount options while adding a primary storage and modify them on a pre-existing primary storage

Open abh1sar opened this issue 10 months ago • 27 comments

Description

This PR adds the UI and CLI support through which NFS mount options can be specified for a primary storage on KVM hosts. The mount options tested for this PR are NFS version and nconnect.

Mount options can be given while adding a Primary storage using the add Zone wizard. It can also be given from the Add Primary storage form. A new field NFS mount options is provided in the respective forms. Mount options can be later modified also from the Primary Storage detail view via the Edit Primary Storage button.

The storage pool needs to be remounted on all the hosts for the new mount option to take affect. So, for a pre-existing storage pool, it has to be in maintenance mode for this functionality to be visible.

NFS mount options are saved in storage_pool_details table and the createStoragePool and updateStoragePool APIs takes the input as a details parameter.

A note on the nconnect option. (https://documentation.suse.com/sles/15-SP5/html/SLES-all/cha-nfs.html)

This option defines the count of TCP connections that the client makes to the NFS server. All the mounts from the same server on the client will have the same nconnect value and all mounts will share the same number of TCP connections.

The nconnect setting is applied only during the first mount process to the particular NFS server. If the same client executes the mount command to the same NFS server, all already established connections will be shared—no new connection will be established. To change the nconnect setting, you have to unmount all clients connections to the particular NFS server. Then you can define a new value of the nconnect option.

So, from CloudStack’s perspective also, the first storage pool created from a NFS server will set the nconnect setting on the hypervisor host corresponding to the server. Specifying a different nconnect mount option while creating a new storage pool from the same server will not change the nconnect setting on the host.

Similarly if there is only one pre-existing storage pool from a give NFS server mounted on the host, modifying the nconnect mount option via CloudStack will change the nconnect setting on that host. If there are more than one storage pools from the same server mounted on a host. Changing the nconnect mount option on one of the storage pools via CloudStack will not do anything. To change the nconnect setting on the host, after modifying nconnect mount option on all storage pools, the host needs to be restarted.

This addresses https://github.com/apache/cloudstack/issues/4482

Doc PR: https://github.com/apache/cloudstack-documentation/pull/396

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • [ ] Major
  • [x] Minor

Bug Severity

  • [ ] BLOCKER
  • [ ] Critical
  • [ ] Major
  • [ ] Minor
  • [ ] Trivial

Screenshots (if appropriate):

Screenshot from 2024-04-24 11-00-39 Primary Storage in Maintenance mode: Screenshot from 2024-04-24 11-29-53 Primary Storage not in Maintenance mode: Screenshot from 2024-04-24 11-30-13

How Has This Been Tested?

Testcase : Mount failing due to incorrect mount options

Add new Primary Storage fails if incorrect mount option is given. Screenshot from 2024-05-02 10-58-32

While updating mount options on a pre-existing pool with incorrect options, cancel maintenance mode will not be able to mount the pool and throw an error. State of the Storage pool is changed to Up from Maintenance before mounting the pool. So, the storage pool will come out of the maintenance mode even in case of the mount error. Screenshot from 2024-05-02 10-59-21

How did you try to break this feature and the system with this change?

abh1sar avatar Apr 19 '24 07:04 abh1sar

@abh1sar given this is a bugfix/improvement related to https://github.com/apache/cloudstack/issues/4482 changed PR base branch to 4.19

rohityadavcloud avatar Apr 19 '24 08:04 rohityadavcloud

@blueorangutan package

harikrishna-patnala avatar Apr 19 '24 08:04 harikrishna-patnala

@harikrishna-patnala a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Apr 19 '24 08:04 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9324

blueorangutan avatar Apr 19 '24 09:04 blueorangutan

@sureshanaparti @harikrishna-patnala @shwstppr have addressed all review comments. Will raise a documentation PR shortly.

abh1sar avatar Apr 24 '24 06:04 abh1sar

@blueorangutan package

sureshanaparti avatar Apr 25 '24 12:04 sureshanaparti

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Apr 25 '24 12:04 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9426

blueorangutan avatar Apr 25 '24 13:04 blueorangutan

@blueorangutan package

abh1sar avatar Apr 30 '24 04:04 abh1sar

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Apr 30 '24 04:04 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9470

blueorangutan avatar Apr 30 '24 05:04 blueorangutan

Codecov Report

Attention: Patch coverage is 59.61538% with 84 lines in your changes missing coverage. Please review.

Project coverage is 14.99%. Comparing base (4d6df58) to head (f683fc9). Report is 2 commits behind head on 4.19.

Files Patch % Lines
.../hypervisor/kvm/storage/LibvirtStorageAdaptor.java 44.73% 13 Missing and 8 partials :warning:
...ain/java/com/cloud/storage/StorageManagerImpl.java 70.90% 8 Missing and 8 partials :warning:
...hypervisor/kvm/resource/LibvirtStoragePoolDef.java 83.33% 1 Missing and 10 partials :warning:
...a/com/cloud/storage/StoragePoolAutomationImpl.java 0.00% 9 Missing :warning:
...ain/java/com/cloud/api/query/QueryManagerImpl.java 0.00% 8 Missing :warning:
...e/cloudstack/api/response/StoragePoolResponse.java 0.00% 6 Missing :warning:
...cycle/CloudStackPrimaryDataStoreLifeCycleImpl.java 28.57% 4 Missing and 1 partial :warning:
...torage/datastore/provider/DefaultHostListener.java 0.00% 3 Missing :warning:
...isor/kvm/resource/LibvirtStoragePoolXMLParser.java 78.57% 1 Missing and 2 partials :warning:
.../com/cloud/agent/api/ModifyStoragePoolCommand.java 0.00% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               4.19    #8947      +/-   ##
============================================
+ Coverage     14.95%   14.99%   +0.03%     
- Complexity    11013    11046      +33     
============================================
  Files          5382     5383       +1     
  Lines        470184   470391     +207     
  Branches      60821    58601    -2220     
============================================
+ Hits          70321    70516     +195     
+ Misses       392075   392044      -31     
- Partials       7788     7831      +43     
Flag Coverage Δ
uitests 4.28% <ø> (-0.01%) :arrow_down:
unittests 15.70% <59.61%> (+0.03%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Apr 30 '24 08:04 codecov-commenter

@blueorangutan package

sureshanaparti avatar May 06 '24 06:05 sureshanaparti

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar May 06 '24 06:05 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 9543

blueorangutan avatar May 06 '24 07:05 blueorangutan

@blueorangutan test rocky8 kvm-rocky8

abh1sar avatar May 06 '24 07:05 abh1sar

@abh1sar a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

blueorangutan avatar May 06 '24 07:05 blueorangutan

[SF] Trillian test result (tid-10167) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 46535 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8947-t10167-kvm-rocky8.zip Smoke tests completed. 128 look OK, 3 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_events_resource Error 423.09 test_events_resource.py
test_01_restore_vm Error 0.22 test_restore_vm.py
test_02_restore_vm_allocated_root Error 0.15 test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown Error 1.24 test_restore_vm.py
test_02_list_cpvm_vm Failure 0.05 test_ssvm.py
test_04_cpvm_internals Failure 0.05 test_ssvm.py
test_12_destroy_cpvm Error 3.25 test_ssvm.py

blueorangutan avatar May 06 '24 21:05 blueorangutan

@blueorangutan package

abh1sar avatar May 08 '24 06:05 abh1sar

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar May 08 '24 06:05 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9574

blueorangutan avatar May 08 '24 07:05 blueorangutan

@blueorangutan test rocky8 kvm-rocky8

abh1sar avatar May 08 '24 08:05 abh1sar

@abh1sar a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

blueorangutan avatar May 08 '24 08:05 blueorangutan

[SF] Trillian test result (tid-10190) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 47086 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8947-t10190-kvm-rocky8.zip Smoke tests completed. 130 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_events_resource Error 424.89 test_events_resource.py

blueorangutan avatar May 08 '24 22:05 blueorangutan

@blueorangutan package

abh1sar avatar May 09 '24 08:05 abh1sar

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar May 09 '24 08:05 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9586

blueorangutan avatar May 09 '24 09:05 blueorangutan

@blueorangutan package

abh1sar avatar May 23 '24 04:05 abh1sar

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar May 23 '24 04:05 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9686

blueorangutan avatar May 23 '24 05:05 blueorangutan