Adding state and execution modules for compliance workflows.

[balavigneshVMware]

trafficstars

Changes include:

1. a state module to invoke dry-run (check compliance) and remediate workflows for compliance controls, for components in VCF.
2. an execution module to invoke the corresponding compliance workflows thru the config-modules interface.
3. utils class to create product contexts.
4. unit tests.

[changes will be merged only after config-module dependency (https://github.com/vmware/vmware-config-modules) is available thru pypi].

[balavigneshVMware]

Should have passing tests before asking for reviews etc. And needs a changelog associated with the PR

Waiting on pypi publish on config-modules, hence marked this PR as "Draft" and haven't marked it "Ready for review" yet. I was not aware github sends notifications even if not "Ready for review".

[dmurphy18]

@balavigneshVMware Got a github email requesting review, so thought you guys had sent it as 'Ready for review'. Wondering why testing is waiting on config-modules to be published, since you can always Mock objects and functions to test the code changes etc. Would have thought that to be standard practice, so not reliant on exterior modules, etc.

[balavigneshVMware]

@dmurphy18, the linux based tests have passed, however docs pipeline is failing with SSL errors reaching out to developer.vmware.com. Do you know who can help with this?

[dmurphy18]

@balavigneshVMware Sorry, our doc writer recently quit, and the SRE that could take a look is away at a conference till the end of the week.. Wondering if you are getting hit by things moving around in the infrastructure, what with the push from AWS to GCP (Google Compute Platform ?).

With failure in SSL Certs, that is rights and infrastructure underneath, perhaps your own SRE can take a look at this, otherwise could ask our SRE when he returns, next week. I would not have rights to mess with Certs.

[balavigneshVMware]

Looks like the docs were migrated to broadcom web portal and the SSL certs might need updating (maybe?). Let me wait for your SRE as we don't have much info on the pipelines.

[dmurphy18]

@ScriptAutomate Can you take a look at this when back from the conference, looks like some certs need updating.

[ScriptAutomate]

Looks like the docs were migrated to broadcom web portal and the SSL certs might need updating (maybe?). Let me wait for your SRE as we don't have much info on the pipelines.

@balavigneshVMware The issue is that all of those links are invalid. If you attempt to go to them, they all merely go to the landing page of the Broadcom developer portal (different domain names, thus the SSL certificate check gets confused).

This leaves you with two options:

1. Fix all the links. The linkcheck tools in Sphinx is to help ensure that links throughout the documentation are accurate. Right now, all of the links flagged in the output are invalid. This means you are directing users/devs to pages that don't actually help them, or are redirects to a different domain / sub-domain.
2. You can choose to keep all of these invalid links (not recommended), and can update the docs/conf.py autodoc config where the linkcheck_ignore is defined and expand the URLs it should ignore:

# <---- Autodoc Config -----------------------------------------------------------------------------------------------

linkcheck_timeout = 10
if not os.environ.get("SKIP_LINKCHECK_IGNORE"):
    linkcheck_ignore = [
        "https://docs.github.com/en/authentication/connecting-to-github-with-ssh",
        "http://pubs.vmware.com",
        "https://developer.vmware.com",
        "https://code.vmware.com"
    ]

Example to test troubleshoot this locally:

# Launch container from root of this repo
docker container run -v $(pwd):/root/salty --rm -it ubuntu:22.04 /bin/bash

# Run these commands in container
cd ~/salty
apt update && apt install python3-pip python3-venv python3-enchant -y
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip nox
nox --force-color -e docs --install-only
nox --force-color -e docs

# Can now view errors, and can modify the links or the docs/conf.py to then
# re-verify

[dmurphy18]

@balavigneshVMware FYI - pytest-skip-markers was recently updated (1.5.2) to correctly support MacOS Arm64, don't think you guys support MacOS but just in case.