No matching distribution found for pulsar-client>=3.1.0

[solvek]

https://console.cloud.google.com/cloud-build/builds;region=global/2a0af7dd-59f1-40f2-8e50-d9508803bbaa?project=harmonia-9dead

It seems this python package is actually available, I was able to install it fine. However it is not installing on google cloud run job.

Actually I was trying to install ChromaDb which depends on pulsar-client It seems the build tried to install distribution type -py3-none-any.whl which is missing for pulsar-client. How to workaround this and use ChromaDb?

[smsohan]

Thanks for letting us know. We don't have access to the build logs. But it'd help if you can share a minimal repro. In that repro, please share the requirements.txt, python version, and any project.toml file if you're using one. Also, please share what command you're using to build, with the cloudbuild.yaml file if you're building on cloud build.

[solvek]

The command to deploy

gcloud run jobs deploy chat-doc \
    --source ./cloud/jobs \
    --tasks 1 \
    --max-retries 1 \
    --region us-central1 \
    --project=harmonia-9dead

Not using any cloudbuild.yaml (at least explicitly)

To reproduce just add a chromadb package dependency.

pyproject.toml file

[project]
name = "chatwise"
version = "0.1"

dependencies = [
  "telethon",
  "langchain >= 0.0.338",
  "google-cloud-aiplatform",
  "chromadb ==0.4.18"
]

It seems here the same issue I have described: https://github.com/GoogleCloudPlatform/buildpacks/issues/365 Initially I had the same error but when I explicitly specified chromadb==0.4.18 I have another erorr in log:

Step #1: [builder] Collecting chroma-hnswlib==0.7.3 (from chromadb>=0.4.18->chatwise==0.1->-r requirements.txt (line 1))
Step #1: [builder]   Downloading chroma-hnswlib-0.7.3.tar.gz (31 kB)
Step #1: [builder]   Installing build dependencies: started
Step #1: [builder]   Installing build dependencies: finished with status 'done'
Step #1: [builder]   Getting requirements to build wheel: started
Step #1: [builder]   Getting requirements to build wheel: finished with status 'done'
Step #1: [builder]   Preparing metadata (pyproject.toml): started
Step #1: [builder]   Preparing metadata (pyproject.toml): finished with status 'done'
Step #1: [builder] Collecting fastapi>=0.95.2 (from chromadb>=0.4.18->chatwise==0.1->-r requirements.txt (line 1))
Step #1: [builder]   Obtaining dependency information for fastapi>=0.95.2 from https://files.pythonhosted.org/packages/f3/4f/0ce34195b63240b6693086496c9bab4ef23999112184399a3e88854c7674/fastapi-0.104.1-py3-none-any.whl.metadata
Step #1: [builder]   Downloading fastapi-0.104.1-py3-none-any.whl.metadata (24 kB)
Step #1: [builder] Collecting uvicorn[standard]>=0.18.3 (from chromadb>=0.4.18->chatwise==0.1->-r requirements.txt (line 1))
Step #1: [builder]   Obtaining dependency information for uvicorn[standard]>=0.18.3 from https://files.pythonhosted.org/packages/7e/17/4b7a76fffa7babf397481040d8aef2725b2b81ae19f1a31b5ca0c17d49e6/uvicorn-0.24.0.post1-py3-none-any.whl.metadata
Step #1: [builder]   Downloading uvicorn-0.24.0.post1-py3-none-any.whl.metadata (6.4 kB)
Step #1: [builder] Collecting posthog>=2.4.0 (from chromadb>=0.4.18->chatwise==0.1->-r requirements.txt (line 1))
Step #1: [builder]   Obtaining dependency information for posthog>=2.4.0 from https://files.pythonhosted.org/packages/a7/73/35758818228c70348be4c3c66a76653c62e894e0e3c3461453c5341ca926/posthog-3.0.2-py2.py3-none-any.whl.metadata
Step #1: [builder]   Downloading posthog-3.0.2-py2.py3-none-any.whl.metadata (2.0 kB)
Step #1: [builder] Collecting typing-extensions>=4.5.0 (from chromadb>=0.4.18->chatwise==0.1->-r requirements.txt (line 1))
Step #1: [builder]   Obtaining dependency information for typing-extensions>=4.5.0 from https://files.pythonhosted.org/packages/24/21/7d397a4b7934ff4028987914ac1044d3b7d52712f30e2ac7a2ae5bc86dd0/typing_extensions-4.8.0-py3-none-any.whl.metadata
Step #1: [builder]   Downloading typing_extensions-4.8.0-py3-none-any.whl.metadata (3.0 kB)
Step #1: [builder] INFO: pip is looking at multiple versions of chromadb to determine which version is compatible with other requirements. This could take a while.
Step #1: [builder] ERROR: Could not find a version that satisfies the requirement pulsar-client>=3.1.0 (from chromadb) (from versions: none)
Step #1: [builder] ERROR: No matching distribution found for pulsar-client>=3.1.0

It seems python 3.12 is used:

Step #1: Status: Downloaded newer image for buildpacksio/lifecycle:0.17.2
Step #1: ===> ANALYZING
Step #1: [analyzer] Timer: Analyzer started at 2023-11-22T16:23:01Z
Step #1: [analyzer] Image with name "us-central1-docker.pkg.dev/harmonia-9dead/cloud-run-source-deploy/chat-doc" not found
Step #1: [analyzer] Timer: Analyzer ran for 65.934µs and ended at 2023-11-22T16:23:01Z
Step #1: ===> DETECTING
Step #1: [detector] Timer: Detector started at 2023-11-22T16:23:02Z
Step #1: [detector] google.python.runtime    0.9.1
Step #1: [detector] google.python.pip        0.9.2
Step #1: [detector] google.config.entrypoint 0.9.0
Step #1: [detector] google.utils.label-image 0.0.2
Step #1: [detector] Timer: Detector ran for 198.880765ms and ended at 2023-11-22T16:23:02Z
Step #1: ===> RESTORING
Step #1: [restorer] Timer: Restorer started at 2023-11-22T16:23:03Z
Step #1: [restorer] Timer: Restorer ran for 944.978µs and ended at 2023-11-22T16:23:03Z
Step #1: ===> BUILDING
Step #1: [builder] Timer: Builder started at 2023-11-22T16:23:03Z
Step #1: [builder] === Python - Runtime ([email protected]) ===
Step #1: [builder] layers path: /layers/google.python.runtime/python
Step #1: [builder] Python version not specified, using the latest available version.
Step #1: [builder] 2023/11/22 16:23:03 [DEBUG] GET https://dl.google.com/runtimes/ubuntu2204/python/version.json
Step #1: [builder] Installing Python v3.12.0.
Step #1: [builder] 2023/11/22 16:23:04 [DEBUG] GET https://dl.google.com/runtimes/ubuntu2204/python/python-3.12.0.tar.gz
Step #1: [builder] Warning: BOM table is deprecated in this buildpack api version, though it remains supported for backwards compatibility. Buildpack authors should write BOM information to <layer>.sbom.<ext>, launch.sbom.<ext>, or build.sbom.<ext>.
Step #1: [builder] Warning: BOM table is deprecated in this buildpack api version, though it remains supported for backwards compatibility. Buildpack authors should write BOM information to <layer>.sbom.<ext>, launch.sbom.<ext>, or build.sbom.<ext>.
Step #1: [builder] Warning: BOM table is deprecated in this buildpack api version, though it remains supported for backwards compatibility. Buildpack authors should write BOM information to <layer>.sbom.<ext>, launch.sbom.<ext>, or build.sbom.<ext>.
Step #1: [builder] === Python - Pip ([email protected]) ===
Step #1: [builder] ***** CACHE MISS: "pip"

[solvek]

Basically I do not need whole chromadb but only chromadb-client package (and this seems to be recommended approach for production). The chromadb server I have deployed as a GCP Run Service. Generally seems to be working now. Still having some issues when using chromadb-client but I think I'll resolve them and write update here.