Adds comet-ml plugin example

[thomasjpfan]

This PR adds a comet-ml example.

To fully run the example, you need to use this ImageSpec, because the plugin is in a PR:

comet_ml_package = (
    "git+https://github.com/thomasjpfan/flytekit.git@"
    "d04d0129f99eb7e7edcffeae04d4f401f49be6d6#subdirectory=plugins/flytekit-comet-ml"
)

image = ImageSpec(
    builder="fast-builder",
    name="unionai",
    apt_packages=["git"],
    packages=[
        "torch==2.3.1",
        "comet-ml==3.43.2",
        "lightning==2.3.0",
        comet_ml_package,
        "unionai==0.1.43",
        "torchvision",
    ],
    registry="ghcr.io/thomasjpfan",
)

https://github.com/flyteorg/flytekit/pull/2550

[thomasjpfan]

@neverett The comet-ml check is passing now: https://github.com/flyteorg/flytesnacks/actions/runs/10182511543/job/28165251450?pr=1702

There are a bunch other tests that are failing tho.

[neverett]

@thomasjpfan can you try merging master in and rerunning?

[fiedlerNr9]

Running the example given in this PR was not working for me with ImageSpec. MisconfigurationException: CometLogger requires either api_key or save_dir during initialization. Execution on Dogfood

I had to add the following command to create a home dir & assign permissions for the flytekit user ->

image = ImageSpec(
    name="comet-ml",
    packages=[
        "torch==2.3.1",
        "comet-ml==3.43.2",
        "lightning==2.3.0",
        "flytekitplugins-comet-ml",
        "torchvision",
    ],
    commands=["mkdir -p /home/flytekit", "chown -R flytekit /home/flytekit"],
    registry=REGISTRY,
)

Anyone experiencing the same?

[thomasjpfan]

For this example to work, it needs ImageSpec(builder="default", ...). The envd builder does not create a home directory for it's user.