packer-plugin-vsphere icon indicating copy to clipboard operation
packer-plugin-vsphere copied to clipboard

Published 20 hours ago • verified publisher icon hashicorp

feat: import ovf template from url in supervisor builder

Open ericvmw opened this issue 9 months ago • 4 comments

Summary

A new feature is being added to Supervisors to expose a new custom resource ContentLibraryItemImportRequest that allows users to import OVF templates to a namespace with writable & import-allowed content library configured on a Supervisor, this enables users to use Packer to orchestrate a complete e2e workflow, from importing their source image from HTTP server, to customize and publish the image to Supervisor clusters.

This change adds configurable optional steps on supervisor builder plugin to allow users to import OVF template to Supervisor namespaces via packer builder vsphere supervisor plugin.

  1. Add import image step to import image from remote URL to the target content library in the specified namespace on the Supervisor by creating ContentLibraryItemImportRequest resource, wait until the import request completes. If keep_input_artifact is set to be true, the import request will be cleaned after it is done. Besides taking the image_name config from existing CreateSource step, this step also support the following optional configs: import_source_url: the remote URL to import OVF template from, must starts with https. import_source_ssl_certificate: the SSL certificate of the remote HTTPS server. import_target_location_name: the import target location (content library that must be writable and allow import). import_target_image_type: optional, the target image type, it defaults to OVF type and only OVF type is supported currently. import_request_name: The name of the ContentLibraryItemImportRequest resource, otherwise it defaults to packer-vsphere-supervisor-import-req-<random-suffix>. watch_import_timeout_sec: The timeout in seconds to wait for the image to be imported, otherwise it defaults to 600. clean_imported_image: optional, whether to clean the image imported in this step. If it is set to true, the imported image will be deleted after source VM is created and becomes ready. Defaults to false.

  2. Add unit tests to cover both import validation and import image steps.

Note that in order to import OVF template from remote URL, the target content library must be configured with the default OVF security policy, and the OVF template must be signed with a certificated that is signed by a CA cert trusted by Content Library Service on vCenter, otherwise the import OVF template image will not be security compliant and cannot be deployed.

Testing

Testing Done:
~/Git/github.com/hashicorp/packer-plugin-vsphere   feat/import-ovf-from-url-on-supervisor
 $ go fmt ./...

~/Git/github.com/hashicorp/packer-plugin-vsphere   feat/import-ovf-from-url-on-supervisor
 $ make generate
2024/05/15 10:36:18 Copying "docs" to ".docs/"
2024/05/15 10:36:18 Replacing @include '...' calls in .docs/ Compiling MDX docs in '.docs' to Markdown in '.web-docs'...

~/Git/github.com/hashicorp/packer-plugin-vsphere   feat/import-ovf-from-url-on-supervisor
 $ make build

~/Git/github.com/hashicorp/packer-plugin-vsphere   feat/import-ovf-from-url-on-supervisor
 $ make test
?   	github.com/hashicorp/packer-plugin-vsphere	[no test files]
?   	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common/testing	[no test files]
?   	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/examples/driver	[no test files]
?   	github.com/hashicorp/packer-plugin-vsphere/version	[no test files]
ok  	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/clone	2.033s
ok  	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common	3.404s
ok  	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/driver	4.862s
ok  	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/iso	2.362s
ok  	github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/supervisor	5.488s
ok  	github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere	1.651s
ok  	github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere-template	2.766s

added unit tests pass:

=== RUN   TestImportImage_Prepare
--- PASS: TestImportImage_Prepare (0.00s)
=== RUN   TestStepImportImage_Run_Skip
2024/05/17 14:58:41 ui: Skipping image import step. Required configurations for the import are not set.
2024/05/17 14:58:41 ui: Skipping image import step. Required configurations for the import are not set.
--- PASS: TestStepImportImage_Run_Skip (0.00s)
=== RUN   TestStepImportImage_Run_Validate
2024/05/17 14:58:41 ui error: error checking required states: missing required state: supervisor_namespace
2024/05/17 14:58:41 ui error: failed to initialize image import: missing required state: supervisor_namespace
2024/05/17 14:58:41 ui error: failed to initialize image import: failed to cast kube_client to type client.WithWatch
2024/05/17 14:58:41 ui: Validating image import request...
2024/05/17 14:58:41 ui error: failed to validate import image configs: import request source url certificate is empty
2024/05/17 14:58:41 ui: Validating image import request...
2024/05/17 14:58:41 ui error: failed to return the content library by name cl-6066c61f7931c5ef9 in namespace test-ns
2024/05/17 14:58:41 ui error: failed to validate import image configs: contentlibraries.imageregistry.vmware.com "cl-6066c61f7931c5ef9" not found
2024/05/17 14:58:41 ui: Validating image import request...
2024/05/17 14:58:41 ui error: failed to validate import image configs: import target content library "cl-6066c61f7931c5ef9" is not writable or does not allow import
2024/05/17 14:58:41 ui: Validating image import request...
2024/05/17 14:58:41 ui error: failed to validate import image configs: image type other is not supported
--- PASS: TestStepImportImage_Run_Validate (0.00s)
=== RUN   TestStepImportImage_Run
2024/05/17 14:58:41 ui: Validating image import request...
2024/05/17 14:58:41 ui: Image import request source and target are valid.
2024/05/17 14:58:41 ui: Importing the source image from https://example.com/example.ovf to cl-6066c61f7931c5ef9.
2024/05/17 14:58:41 ui: Creating ContentLibraryItemImportRequest object test-req-name in namespace test-ns.
2024/05/17 14:58:41 ui: Successfully created the ContentLibraryItemImportRequest object test-req-name.
2024/05/17 14:58:42 ui: Successfully imported the image as a content library item &{"imageregistry.vmware.com/v1alpha1" "ContentLibraryItem" "clitem-xxx"}.
2024/05/17 14:58:42 ui: Finished importing the image from https://example.com/example.ovf to cl-6066c61f7931c5ef9.
--- PASS: TestStepImportImage_Run (1.00s)
=== RUN   TestStepImportImage_Cleanup
2024/05/17 14:58:42 ui: Skipping clean up of the ContentLibraryItemImportRequest object as specified in config.
2024/05/17 14:58:42 ui: Deleting the ContentLibraryItemImportRequest object test-req-name from Supervisor cluster.
2024/05/17 14:58:42 ui: Successfully deleted the ContentLibraryItemImportRequest object.
--- PASS: TestStepImportImage_Cleanup (0.00s)

$ make testacc
all acceptance tests pass except some tests under github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/iso, which doesn't look related to the change.

Supervisor packer build pass with import_source URL: https://wp-content-pstg.broadcom.com/vmimagesvc/ubuntu-jammy-22.04-cloudimg-signed/ubuntu-jammy-22.04-cloudimg-signed.ovf

packer build --var-file tmp/packer-example.json tmp/packer-example.pkr.hcl vsphere-supervisor.vm: 

==> vsphere-supervisor.vm: Creating temporary RSA SSH key for instance...
    vsphere-supervisor.vm: Connecting to Supervisor cluster...
    vsphere-supervisor.vm: Successfully connected to Supervisor cluster
    vsphere-supervisor.vm: Validating VM publish location...
    vsphere-supervisor.vm: VM publish location is valid
    vsphere-supervisor.vm: Validating image import request...
    vsphere-supervisor.vm: Image import request source and target are valid
    vsphere-supervisor.vm: Importing the source image from https://wp-content-pstg.broadcom.com/vmimagesvc/ubuntu-jammy-22.04-cloudimg-signed/ubuntu-jammy-22.04-cloudimg-signed.ovf to cl-6066c61f7931c5ef9
    vsphere-supervisor.vm: Creating a ContentLibraryItemImportRequest object <sensitive>-vsphere-supervisor-import-req-71b5a
    vsphere-supervisor.vm: Successfully created the ContentLibraryItemImportRequest object <sensitive>-vsphere-supervisor-import-req-71b5a
    vsphere-supervisor.vm: Waiting for the image import request to complete...
    vsphere-supervisor.vm: Waiting for the image import request to complete...
    vsphere-supervisor.vm: Successfully imported the image as library item &{"imageregistry.<sensitive>.com/v1alpha1" "ContentLibraryItem" "clitem-c950e9d6dc3b57ee4"}
    vsphere-supervisor.vm: Finished importing the image from https://wp-content-pstg.broadcom.com/vmimagesvc/ubuntu-jammy-22.04-cloudimg-signed/ubuntu-jammy-22.04-cloudimg-signed.ovf to "cl-6066c61f7931c5ef9"
    vsphere-supervisor.vm: Creating required source objects in Supervisor cluster...
    vsphere-supervisor.vm: Creating a K8s Secret object for providing source VM bootstrap data...
    vsphere-supervisor.vm: Using default cloud-init user data as the 'bootstrap_data_file' is not specified
    vsphere-supervisor.vm: Successfully created the K8s Secret object
    vsphere-supervisor.vm: Creating a source VirtualMachine object
    vsphere-supervisor.vm: Successfully created the VirtualMachine object
    vsphere-supervisor.vm: Creating a VirtualMachineService object for network connection
    vsphere-supervisor.vm: Successfully created the VirtualMachineService object
    vsphere-supervisor.vm: Finished creating all required source objects in Supervisor cluster
    vsphere-supervisor.vm: Waiting for the source VM to be powered-on and accessible...
    vsphere-supervisor.vm: Source VM is NOT powered-on yet, continue watching...
    vsphere-supervisor.vm: Source VM is NOT powered-on yet, continue watching...
    vsphere-supervisor.vm: Source VM is powered-on, waiting for an IP to be assigned...
    vsphere-supervisor.vm: Source VM is powered-on, waiting for an IP to be assigned...
    vsphere-supervisor.vm: Successfully obtained the source VM IP: 192.168.128.189
    vsphere-supervisor.vm: Getting source VM ingress IP from the VMService object
    vsphere-supervisor.vm: Successfully retrieved the source VM ingress IP: 192.168.0.4
    vsphere-supervisor.vm: Source VM is now ready in Supervisor cluster
==> vsphere-supervisor.vm: Using SSH communicator to connect: 192.168.0.4 ==> vsphere-supervisor.vm: Waiting for SSH to become available... ==> vsphere-supervisor.vm: Connected to SSH!
==> vsphere-supervisor.vm: Provisioning with shell script: /var/folders/gc/ptc0zb0507b05r97rqnb5cym0000gq/T/<sensitive>-shell1862650776
    vsphere-supervisor.vm: Publishing the source VM to "cl-6066c61f7931c5ef9"
    vsphere-supervisor.vm: Creating a VirtualMachinePublishRequest object
    vsphere-supervisor.vm: Successfully created the VirtualMachinePublishRequest object
    vsphere-supervisor.vm: Waiting for the VM publish request to complete...
    vsphere-supervisor.vm: Waiting for the VM publish request to complete...
    vsphere-supervisor.vm: Successfully published the VM to image "vmi-2569621448a150e8a"
    vsphere-supervisor.vm: Finished publishing the source VM
    vsphere-supervisor.vm: Deleting the VirtualMachinePublishRequest object from Supervisor cluster
    vsphere-supervisor.vm: Successfully deleted the VirtualMachinePublishRequest object
    vsphere-supervisor.vm: Deleting the imported ContentLibraryItem object clitem-19fab321189b7b177 in namespace test-k8s-service.
    vsphere-supervisor.vm: Successfully deleted the ContentLibraryItem object clitem-19fab321189b7b177 in namespace test-k8s-service.
    vsphere-supervisor.vm: Deleting the VirtualMachineService object from Supervisor cluster
    vsphere-supervisor.vm: Successfully deleted the VirtualMachineService object
    vsphere-supervisor.vm: Deleting the VirtualMachine object from Supervisor cluster
    vsphere-supervisor.vm: Successfully deleted the VirtualMachine object
    vsphere-supervisor.vm: Deleting the K8s Secret object from Supervisor cluster
    vsphere-supervisor.vm: Successfully deleted the K8s Secret object
    vsphere-supervisor.vm: Deleting the ContentLibraryItemImportRequest object <sensitive>-vsphere-supervisor-import-req-71b5a from Supervisor cluster
    vsphere-supervisor.vm: Successfully deleted the ContentLibraryItemImportRequest object
    vsphere-supervisor.vm: Build 'vsphere-supervisor' finished successfully.
Build 'vsphere-supervisor.vm' finished after 5 minutes 3 seconds.

Supervisor packer build pass without setting import_source_url (use existing image instead)

Reference

Closes #434

ericvmw avatar May 15 '24 18:05 ericvmw