How to run conda based flows in a push workpool

[not-Karot]

First check

• [X] I added a descriptive title to this issue.
• [X] I used GitHub search to find a similar request and didn't find it 😇

Describe the issue

Deploying flows within GCR push workpolls on a conda based Docker image ends with

/opt/conda/bin/python: Error while finding module specification for 'prefect.engine' (ModuleNotFoundError: No module named 'prefect')

There is no page in prefect documentation describing how to run conda based image and nothing seems to run as expected.

Describe the proposed change

Add tutorials, examples and guidelines on how to manage prefect conda images.

Additional context

No response

[jakekaplan]

Hi @not-Karot thanks for filing this. Can you give some more information about what you're running? Which image specifically etc.?

[not-Karot]

Yes, sure @jakekaplan

This is my dockerfile:

FROM prefecthq/prefect:2-python3.10-conda

COPY environment.yml .
COPY pyproject.toml .
COPY poetry.lock .
COPY setup.cfg .
COPY setup.py .
COPY my_code .

RUN apt-get update && apt-get install -y libarchive13

RUN conda install -c conda-forge mamba
RUN mamba env update --prefix /opt/conda/envs/prefect -f environment.yml
RUN conda activate prefect
RUN poetry install

ENV PYTHONUNBUFFERED True

COPY flows/ /opt/prefect/flows/

and this is my prefect.yaml.

name: my-deployments
prefect-version: 2.14.13

# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: us-central1-docker.pkg.dev/feelicity-dev/my-repository/my-image
    tag: latest
    dockerfile: Dockerfile
    platform: linux/amd64

# push section allows you to manage if and how this project is uploaded to remote locations
push:
- prefect_docker.deployments.steps.push_docker_image:
    requires: prefect-docker>=0.3.1
    image_name: '{{ build_image.image_name }}'
    tag: '{{ build_image.tag }}'

# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/my_code

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: flow-main
  version: 1
  tags: [first, sample]
  description: First deployment
  schedule: {}
  entrypoint: flows/my_flow.py:my_flow0
  parameters: { }
  work_pool:
    name: my-pool
    job_variables:
      image: '{{ build_image.image }}'

my-poll is a pool built with prefect cli command to create a push work poll on google cloud run.

I can't make it work and flow does not run on gcr.

If i change dockerimage building it from a pip based image, everything works well.