harbor icon indicating copy to clipboard operation
harbor copied to clipboard
verified publisher icon av

Fail to download JSR package

Open shenhai-ran opened this issue 10 months ago • 6 comments

Hello,

I just updated to the latest main branch and I have trouble to get Harbor started, and the error message is like this:

Download https://jsr.io/@std/yaml/meta.json
Download https://jsr.io/@std/collections/meta.json
Download https://jsr.io/@std/fmt/meta.json
error: JSR package manifest for '@std/yaml' failed to load. Import 'https://jsr.io/@std/yaml/meta.json' failed.
    at file:///mnt/data/apps/harbor/routines/mergeComposeFiles.js:1:23
exit status 1

It looks like the new routines functions require some connect with jsr.io and I have tried curl https://jsr.io/@std/yaml/meta.json it produce some result:

{
  "scope": "std",
  "name": "yaml",
  "latest": "1.0.5",
  "versions": {
    "1.0.2": {},
    "0.210.0": {},
    "0.209.0": {},
    "0.202.0": {},
    "0.204.0": {},
    "0.211.0": {},
    "0.224.1": {},
    "1.0.1": {},
    "0.222.1": {},
    "1.0.5": {},
    "0.220.1": {},
    "0.218.1": {},
    "0.199.0": {},
    "1.0.3": {},
    "1.0.4": {},
    "0.205.0": {},
    "0.217.0": {},
    "1.0.0-rc.3": {},
    "0.224.3": {},
    "0.221.0": {},
    "0.206.0": {},
    "0.196.0": {},
    "0.218.0": {},
    "0.224.2": {},
    "0.224.0": {},
    "0.216.0": {},
    "0.214.0": {},
    "0.215.0": {},
    "0.198.0": {},
    "0.208.0": {},
    "0.197.0": {},
    "0.219.0": {},
    "1.0.0-rc.1": {},
    "0.212.0": {},
    "0.213.1": {},
    "0.222.0": {},
    "0.207.0": {},
    "0.201.0": {},
    "0.219.1": {},
    "0.200.0": {},
    "0.218.2": {},
    "1.0.0-rc.2": {},
    "0.223.0": {},
    "0.213.0": {},
    "0.203.0": {},
    "1.0.0-rc.4": {},
    "1.0.0": {}
  }

I am behind the proxy and has self-signed certificate and I don't know if it matters. Thanks for your help!

Regards,

shenhai-ran avatar Mar 04 '25 09:03 shenhai-ran

Hi, thanks for the detailed report and sorry that the update wasn't smooth in your instance!

Yes, it's related to the new routines functionality that has lazy/dynamic dependency install from JSR during the cold start.

  1. From the looks of it - you do have access to JSR from behind your proxy (granted docker traffic is routed in the same way as the traffic from your host where curl request was made).

  2. Another observation - there's only error for @std/yaml, but not other packages. There's a possibility that it was this temporary problem: https://github.com/denoland/deno/issues/24073 in which instance it should've been resolved within a few minutes. Could you please re-try starting the services?

  3. As another way to debug, you can try this installation standalone to see if it'll work:

docker run denoland/deno:distroless install jsr:@std/yaml

If this command is passing, then the issue is not the registry per se, but me doing something wrong with launching the routine 😅

Finally, If none of the above helps - it's possible to return to the legacy CLI as a workaround:

harbor config set legacy.cli true

P.S. I tried to find any instructions on using JSR behind a proxy/firewall, but there's nothing indicating that they'd prevent such use.

av avatar Mar 04 '25 13:03 av

Hi,

Thanks for your quick feedback!

  1. Actually this might be the problem, I tried to attach harbor.ollama container, and has trouble with curl connection:
# curl https://jsr.io/@std/yaml/meta.json
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

For other harbor services require network connection, I have already setup my certificate and curl works fine.

  1. Standalone installation doesn't work either, not surprisingly.
Created deno.json configuration file.
error: jsr:@std/yaml was not found.
exit status 1

I guess there should be some docker services for the new routines, where can I configure them, so I can setup certificate as well?

Thanks!

shenhai-ran avatar Mar 06 '25 08:03 shenhai-ran

there should be some docker services for the new routines

Yes, but it's fully inline in the CLI here, so any modifications of the volumes needs to happen there, I think another issue would be that it's a distroless image, so it might not have means for a typical SSL cert mount/override

av avatar Apr 19 '25 14:04 av

I have the same issue but without a proxy. Only fallback to legacy cli let me start harbor.

boriskroeger avatar Jun 09 '25 12:06 boriskroeger

doesn't work here either.
ONly way to get it running is legacy cli.

kundeng avatar Aug 11 '25 17:08 kundeng

Issue Description: Harbor JSR Import Failure on Linux

Problem

Harbor's harbor ls command fails on Linux with JSR package import errors: error: JSR package manifest for '@std/yaml' failed to load. Import 'https://jsr.io/@std/yaml/meta.json' failed.

Logical Sequence of Events

  1. Harbor's Architecture: Harbor uses a containerized Deno runtime for TypeScript routines - Main CLI is a bash script (/home/kundeng/.local/bin/harbor) - TypeScript routines live in ~/.harbor/routines/ (like mergeComposeFiles.ts) - Routines run inside denoland/deno:distroless container via Docker
  2. Container Execution Flow: harbor ls → get_services() → compose_with_options() → run_routine mergeComposeFiles → Docker container execution
  3. Docker Run Command (from run_routine function): docker run --rm
    -v "$harbor_home:$harbor_home"
    -v harbor-deno-cache:/deno-dir:rw
    -w "$harbor_home"
    -e "HARBOR_LOG_LEVEL=$default_log_level"
    denoland/deno:distroless run -A --unstable-sloppy-imports
    /path/to/mergeComposeFiles.ts
  4. Network Configuration Issue: - Default Docker networking: Container uses bridge network (isolated) - No explicit network flags: Container cannot reach external registries - JSR dependency resolution: Deno tries to fetch https://jsr.io/@std/yaml/meta.json but times out

Mac vs Linux Difference

Why it works on Mac but not Linux:

  1. Docker Desktop vs Docker Engine: - Mac (Docker Desktop): More permissive default networking, often allows outbound connections - Linux (Docker Engine): Stricter bridge network isolation by default
  2. Network Stack Differences: - Mac: Docker Desktop creates a more transparent network bridge - Linux: Default bridge network has stricter outbound policies
  3. Corporate/Firewall Environment: - Linux box might be in a more restricted network environment - Mac might have different DNS/proxy configurations

Root Cause

Harbor's container execution lacks proper network configuration for external registry access. The container runs in Docker's default bridge network which prevents JSR registry downloads.

Temporary Workaround vs Real Fix

Current Workaround: docker run --network=host ... deno cache --reload mergeComposeFiles.ts Pre-caches dependencies using host networking, but this is a one-time manual fix.

Proper Fix for PR: Harbor's run_routine function needs to add network configuration:

In routines/run_routine function, line 349-355:

docker run --rm
--network=host \ # Add this line -v "$harbor_home:$harbor_home"
-v harbor-deno-cache:/deno-dir:rw
# ... rest unchanged

Or alternatively, configure proper DNS/networking for the bridge network to allow external registry access.

The real fix ensures Harbor can always download dependencies when needed, rather than requiring manual pre-caching.

kundeng avatar Aug 11 '25 19:08 kundeng