Retrieve_unpackaged unusable in MetaDeploy

[yippie]

Describe the bug

The retrieve_unpackaged task is messing with the options directly which causes issues with Metadeploy.

In _init_options, the package_xml option, which is supposed to be a file path, gets overwritten to the content of the file.

When compiling tasks for Metadeploy, the _init_options appears to be run and the modified options are then saved as the step options. When run in MetaDeploy, it then runs _init_options again and attempts to use the package.xml file content to find the new file path.

Reproduction steps

1. Connect to Metadeploy
2. Have a plan that includes the retrieve_unpackaged as a step
3. Upload the plan
4. Attempt to run the plan in Metadeploy. Run fails during initialization of the retrieve_unpackaged step

Your CumulusCI and Python versions

CumulusCI version: 3.74.0 (/Library/Frameworks/Python.framework/Versions/3.10/bin/cci) Python version: 3.10.9 (/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10)

Operating System

macOS Ventura

Windows environment

No response

CumulusCI installation method

pipx

Error Gist

No response

Additional information

No response

[davidmreed]

I'm not surprised this doesn't work. Can you share details about what you're aiming to achieve by running retrieve_unpackaged in MetaDeploy? Since that task is intended for use during development, it's likely never been tested as part of a "frozen" MetaDeploy plan.

[yippie]

Short answer, backups.

Long answer, we made choice (mistake?) of using unlocked packages instead of managed packages (only 1gp were available at the time) for our multi-package product. The downside of unlocked packages is that, unlike managed packages, picklists are controlled by the package, not the subscriber, so upgrades will overwrite any customer picklist changes.

The hackaround we are doing, backup the picklists before upgrade and re-deploy them after. This is where retrieve_unpackaged comes from, we have a specific list of picklists to back up. As a bonus, we actually take the backup, zip it up, and upload it as a static resource so it can be separately restored if needed, which also uses retrieve_unpackage to download the backup.