openvsx icon indicating copy to clipboard operation
openvsx copied to clipboard
verified publisher icon eclipse

How to configure mirror mode?

Open putradimas opened this issue 7 months ago • 4 comments

I want to prepare a vsix server for an airgapped VMs with vscode IDE and found that openvsx might be suitable for this. But I can't seem to work out how to configure it.

So, my questions:

  1. How to setup a partial mirror of vsix extensions by providing list of extensions to download? Preferably with version selector instead of mirroring the whole versions.
  2. Is there any better way for an airgapped deployment?

I'm using openvsx 0.25.0 running on docker

The config I'm using is copied from this https://github.com/gitpod-io/openvsx/blob/df16f9846aa00641d9bde301fe6d1e49bdc493cb/server/src/dev/resources/application-mirror.yml

ovsx:
  data:
    mirror:
      enabled: true
      server-url: https://open-vsx.org
      requests-per-second: 5
      user-name: mirror_user
      schedule: '*/1 * * * *'
      read-only:
        disallowed-methods: POST, PUT, PATCH, DELETE
        allowed-endpoints: /vscode/gallery/extensionquery, /admin/update-search-index,

Here's the log snippet:

2025-06-03T18:25:55.037Z ERROR [openvsx-server,,] 1 --- [openvsx-server] [ool-11-thread-1] [                                                 ] o.e.openvsx.UpstreamRegistryService      : GET /api/Turiiya/german-scroll

org.springframework.web.client.ResourceAccessException: I/O error on GET request for "/api/Turiiya/german-scroll": Target host is not specified
    at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:915) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:895) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:811) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:413) ~[spring-web-6.1.18.jar:6.1.18]
    at org.eclipse.openvsx.UpstreamRegistryService.getExtension(UpstreamRegistryService.java:118) ~[classes/:na]
    at jdk.internal.reflect.GeneratedMethodAccessor47.invoke(Unknown Source) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:355) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.eclipse.openvsx.mirror.aop.UpstreamRegistryServiceAspect.rateLimitMethodCall(UpstreamRegistryServiceAspect.java:38) ~[classes/:na]
    at jdk.internal.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:641) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:631) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:720) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.eclipse.openvsx.UpstreamRegistryService$$SpringCGLIB$$0.getExtension(<generated>) ~[classes/:na]
    at org.eclipse.openvsx.mirror.MirrorExtensionService.mirrorExtension(MirrorExtensionService.java:79) ~[classes/:na]
    at org.eclipse.openvsx.mirror.DataMirrorJobRequestHandler.processUrls(DataMirrorJobRequestHandler.java:124) ~[classes/:na]
    at org.eclipse.openvsx.mirror.DataMirrorJobRequestHandler.run(DataMirrorJobRequestHandler.java:97) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner$BackgroundJobWorker.invokeJobMethod(AbstractBackgroundJobRunner.java:65) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner$BackgroundJobWorker.run(AbstractBackgroundJobRunner.java:39) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner.run(AbstractBackgroundJobRunner.java:21) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.runActualJob(BackgroundJobPerformer.java:95) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.performJob(BackgroundJobPerformer.java:68) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.run(BackgroundJobPerformer.java:46) ~[jobrunr-7.5.0.jar:7.5.0]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]
Caused by: org.apache.hc.client5.http.ClientProtocolException: Target host is not specified
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:99) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:70) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66) ~[spring-web-6.1.18.jar:6.1.18]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:889) ~[spring-web-6.1.18.jar:6.1.18]
    ... 44 common frames omitted
Caused by: org.apache.hc.core5.http.ProtocolException: Target host is not specified
    at org.apache.hc.client5.http.impl.routing.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:66) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.determineRoute(InternalHttpClient.java:120) ~[httpclient5-5.2.1.jar:5.2.1]
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:158) ~[httpclient5-5.2.1.jar:5.2.1]
    ... 51 common frames omitted

2025-06-03T18:25:55.040Z ERROR [openvsx-server,,] 1 --- [openvsx-server] [ool-11-thread-1] [                                                 ] o.e.o.m.DataMirrorJobRequestHandler      : failed to mirror Turiiya.german-scroll

org.eclipse.openvsx.util.NotFoundException: null
    at org.eclipse.openvsx.UpstreamRegistryService.getExtension(UpstreamRegistryService.java:132) ~[classes/:na]
    at jdk.internal.reflect.GeneratedMethodAccessor47.invoke(Unknown Source) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:355) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.eclipse.openvsx.mirror.aop.UpstreamRegistryServiceAspect.rateLimitMethodCall(UpstreamRegistryServiceAspect.java:38) ~[classes/:na]
    at jdk.internal.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:641) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:631) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:720) ~[spring-aop-6.1.18.jar:6.1.18]
    at org.eclipse.openvsx.UpstreamRegistryService$$SpringCGLIB$$0.getExtension(<generated>) ~[classes/:na]
    at org.eclipse.openvsx.mirror.MirrorExtensionService.mirrorExtension(MirrorExtensionService.java:79) ~[classes/:na]
    at org.eclipse.openvsx.mirror.DataMirrorJobRequestHandler.processUrls(DataMirrorJobRequestHandler.java:124) ~[classes/:na]
    at org.eclipse.openvsx.mirror.DataMirrorJobRequestHandler.run(DataMirrorJobRequestHandler.java:97) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner$BackgroundJobWorker.invokeJobMethod(AbstractBackgroundJobRunner.java:65) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner$BackgroundJobWorker.run(AbstractBackgroundJobRunner.java:39) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.runner.AbstractBackgroundJobRunner.run(AbstractBackgroundJobRunner.java:21) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.runActualJob(BackgroundJobPerformer.java:95) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.performJob(BackgroundJobPerformer.java:68) ~[jobrunr-7.5.0.jar:7.5.0]
    at org.jobrunr.server.BackgroundJobPerformer.run(BackgroundJobPerformer.java:46) ~[jobrunr-7.5.0.jar:7.5.0]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]

putradimas avatar Jun 03 '25 18:06 putradimas

What value does ovsx.upstream.url have?

amvanbaren avatar Jun 04 '25 07:06 amvanbaren

I use the same exact file as the linked config file https://github.com/gitpod-io/openvsx/blob/df16f9846aa00641d9bde301fe6d1e49bdc493cb/server/src/dev/resources/application-mirror.yml#L23

ovsx:
    upstream:
        url: https://open-vsx.org

putradimas avatar Jun 05 '25 02:06 putradimas

I have encountered the exact same issue using the mirror configuration as described above. My OpenVSX instance runs in Kubernetes with mirror mode enabled, and I’m seeing org.springframework.web.client.ResourceAccessException: Target host is not specified in the logs.

It looks like the mirror process tries to use a relative path (like /api/...) instead of a full URL, so Java RestTemplate fails with “Target host is not specified.”

Environment:

OpenVSX version: [v0.27.0]

Deployment: [Kubernetes]

JVM: [e.g. OpenJDK 17]

Please confirm if this is a known bug, and if there are any workarounds or recommended image versions to avoid this issue.

Thanks!

zhongyouyang avatar Aug 04 '25 14:08 zhongyouyang