Issue importing generated crd with python 3.11

[jaxxstorm]

What happened?

I have previously been able to get this to work, but on python 3.11 I get the following error:

Diagnostics:
  pulumi:pulumi:Stack (varnish_crd-dev):
    error: Program failed with an unhandled exception:
    Traceback (most recent call last):
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/__main__.py", line 5, in <module>
        import crds.python.pulumi_crds as varnish
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/crds/python/pulumi_crds/__init__.py", line 5, in <module>
        from . import _utilities
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/crds/python/pulumi_crds/_utilities.py", line 94, in <module>
        _version = _get_semver_version()
                   ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/crds/python/pulumi_crds/_utilities.py", line 73, in _get_semver_version
        pep440_version_string = pkg_resources.require(root_package)[0].version
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 966, in require
        needed = self.resolve(parse_requirements(requirements))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 827, in resolve
        dist = self._resolve_dist(
               ^^^^^^^^^^^^^^^^^^^
      File "/Users/lbriggs/src/github/jaxxstorm/pulumi-examples/python/kubernetes/varnish_crd/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 868, in _resolve_dist
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'crds' distribution was not found and is required by the application

Expected Behavior

I can provision a custom resource

Steps to reproduce

Download a CRD:

wget https://raw.githubusercontent.com/IBM/varnish-operator/main/varnish-operator/crds/varnishcluster.yaml

Generate the default types

crd2pulumi -p varnishcluster.yaml

Import and use:

import crds.python.pulumi_crds as varnish


varnish_cluster = varnish.caching.v1alpha1.VarnishCluster(
    "example",
    spec=varnish.caching.v1alpha1.VarnishClusterSpecArgs(
        vcl=varnish.caching.v1alpha1.VarnishClusterSpecVclArgs(
            config_map_name="example-vcl",
            entrypoint="entrypoint.vcl",
        ),
        backend=varnish.caching.v1alpha1.VarnishClusterSpecBackendArgs(
            port=8080,
        )
    )
)

Output of pulumi about

CLI
Version      3.67.0
Go Version   go1.20.4
Go Compiler  gc

Plugins
NAME        VERSION
kubernetes  3.27.1
python      3.11.3

Host
OS       darwin
Version  13.3.1
Arch     arm64

This project is written in python: executable='/opt/homebrew/bin/python3' version='3.11.3
'

Current Stack: lbrlabs58/varnish_crd/dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/lbrlabs58
User           jaxxstorm
Organizations  jaxxstorm, lbrlabs, demo

Dependencies:
NAME               VERSION
pip                23.1.2
pulumi-kubernetes  3.27.1
setuptools         67.7.2
wheel              0.40.0

Pulumi locates its logs in /var/folders/rk/2c715ngd14jc4c7d3659m7pm0000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Hi @jaxxstorm thanks for reporting the issue here. I took a quick look. It seems that _get_semver_version() makes certain assumptions about how the package is installed and it's coming from pulumi/pulumi SDK Python generator, it's optimized for providers and is not working quite as expected here. I'm testing on Python 3.9 but I think it applies to all versions.

I found that there is a workaround though. If you create a python Project pulumi new python then you can reference crd2pulumi generated code into the Pulumi project's venv like so:

./venv/bin/python -m pip install -e ../crds/python/pulumi_crds                                         

Then you can import this code as follows:

import pulumi_crds as varnish

varnish_cluster = varnish.caching.v1alpha1.VarnishCluster(
    "example",
    spec=varnish.caching.v1alpha1.VarnishClusterSpecArgs(
        service=None,
        vcl=varnish.caching.v1alpha1.VarnishClusterSpecVclArgs(
            config_map_name="example-vcl",
            entrypoint_file_name="entrypoint.vcl",
        ),
        backend=varnish.caching.v1alpha1.VarnishClusterSpecBackendArgs(
            port=8080,
            selector={}
        )
    )
)

I had to correct some arguments - I didn't get through all the errors unfortunately but it got me to the following:

    error: Could not automatically download and install resource plugin 'pulumi-resource-kubernetes' at version v0.0.0, install the plugin using `pulumi plugin install resource kubernetes v0.0.0`: error downloading provider kubernetes to file: failed to download plugin: kubernetes-0.0.0: 403 HTTP error fetching plugin from https://get.pulumi.com/releases/plugins/pulumi-resource-kubernetes-v0.0.0-darwin-amd64.tar.gz

[danmur]

A workaround seems to be to give the generated package a version. Definitely seems like a bug though.

[cleverguy25]

Added to epic https://github.com/pulumi/home/issues/3431