Add lite configuration environment

[pditommaso]

trafficstars

This PR introduces a configuration profile named lite. When applied, it disables the functionalities that relies on the use of Kubernetes cluster and a shared file system for carrying out the execution compute intensive jobs, and streamline the service installation.

As a side effect, when enabling this option, the following features are not available:

• Container Freeze
• Container Build service
• Container Mirror service
• Container Security scanning
• Container blobs caching

[munishchouhan]

tested for build:

% wave --conda-package bwa --wave-endpoint http://localhost:9090
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}
% wave -s -f Singularity --wave-endpoint http://localhost:9090 --tower-endpoint http://localhost:8000/api --tower-token <TOKEN> --freeze --build-repo docker.io/hrma017/sing --platform linux/arm64
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}
(base) munish.chouhan@Munishs-MacBook-Pro singularity % wave -f Dockerfile --wave-endpoint http://localhost:9090 --tower-endpoint http://localhost:8000/api --tower-token <TOKEN> --freeze --build-repo docker.io/hrma017/sing --platform linux/arm64
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}
(base) munish.chouhan@Munishs-MacBook-Pro singularity % wave -f Dockerfile --wave-endpoint http://localhost:9090                                                               
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}

[munishchouhan]

tested as proxy:

 % wave -i ubuntu --wave-endpoint http://localhost:9090
wave.eu.ngrok.io/wt/c21012677872/library/ubuntu:latest

[munishchouhan]

tested mirroring:

 %  wave -i quay.io/biocontainers/bwa:0.7.15--0 --mirror --build-repository docker.io/hrma017 --tower-token <TOKEN> --wave-endpoint http://localhost:9090 --tower-endpoint http://localhost:8000/api
Wave invalid response: [501] {"message":"Mirror service is not enabled - Check Wave configuration setting 'wave.mirror.enabled' - HTTP status=501 - response=-"}

[munishchouhan]

Tested augmentation:

 % wave -i ubuntu --config-file https://fusionfs.seqera.io/releases/v2.5-amd64.json --wave-endpoint http://localhost:9090
wave.eu.ngrok.io/wt/504999fe3eb4/library/ubuntu:latest

[munishchouhan]

@pditommaso, should we add a message for scan mode too?

 % wave -i ubuntu --scan-mode required --await --wave-endpoint http://localhost:9090
wave.eu.ngrok.io/wt/0d4b4710ca43/library/ubuntu:latest

[pditommaso]

Currently what error is returning?

[munishchouhan]

Currently what error is returning?

Currently no error, it return the image URL and scan does not get triggered

[pditommaso]

This edab8b7 should solve

[munishchouhan]

This edab8b7 should solve

moved the check to a upper method because scan can be requested without build

[munishchouhan]

tested scan mode:

% wave -i ubuntu --scan-mode required  --wave-endpoint http://localhost:9090
Wave invalid response: [501] {"message":"Security scan service is not enabled - Check Wave configuration setting 'wave.scan.enabled' - HTTP status=501 - response=-"}

[gitguardian[bot]]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
16707457	Triggered	Generic Password	83d1833af5ad9f53bcde0bdbb2d0c7a869440bf8	lite/config.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[pditommaso]

@munishchouhan Let's make a pre-release version and deploy to stage

[munishchouhan]

@munishchouhan Let's make a pre-release version and deploy to stage

lite + postgres?

[pditommaso]

Yes (enabling only postgres in our deployment)

[munishchouhan]

@pditommaso there is one issue in lite is that it still required build-workspace to run because of this method in scanconfig https://github.com/seqeralabs/wave/blob/ac81a5793c0887676a1c9ffc08e423ca67294cc8/src/main/groovy/io/seqera/wave/configuration/ScanConfig.groovy#L114

Should we move it to ContainerScanServiceImpl?

[munishchouhan]

@pditommaso there is one issue in lite is that it still required build-workspace to run because of this method in scanconfig

https://github.com/seqeralabs/wave/blob/ac81a5793c0887676a1c9ffc08e423ca67294cc8/src/main/groovy/io/seqera/wave/configuration/ScanConfig.groovy#L114

Should we move it to ContainerScanServiceImpl?

ok instead of it, I have made build workspace nullable which will solve the issue

[munishchouhan]

Tested:

(base) munish.chouhan@Munishs-MacBook-Pro ~ % wave -i ubuntu --wave-endpoint https://wave.eu.ngrok.io
wave.eu.ngrok.io/wt/aab7725fa1e7/library/ubuntu:latest
(base) munish.chouhan@Munishs-MacBook-Pro ~ % docker pull wave.eu.ngrok.io/wt/aab7725fa1e7/library/ubuntu:latest
latest: Pulling from wt/aab7725fa1e7/library/ubuntu
2f074dc76c5d: Pull complete
Digest: sha256:6015f66923d7afbc53558d7ccffd325d43b4e249f41a6e93eef074c9505d2233
Status: Downloaded newer image for wave.eu.ngrok.io/wt/aab7725fa1e7/library/ubuntu:latest
wave.eu.ngrok.io/wt/aab7725fa1e7/library/ubuntu:latest

% wave --conda-package bwa  --wave-endpoint https://wave.eu.ngrok.io
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}

[munishchouhan]

[munishchouhan]

tested in local machine

(base) munish.chouhan@Munishs-MacBook-Pro ~ % wave -i ubuntu --config-file https://fusionfs.seqera.io/releases/v2.4.13-arm64.json --wave-endpoint http://localhost:9090
334e-84-127-154-163.ngrok-free.app/wt/b56717475bcc/library/ubuntu:latest
(base) munish.chouhan@Munishs-MacBook-Pro ~ % docker pull 334e-84-127-154-163.ngrok-free.app/wt/b56717475bcc/library/ubuntu:latest
latest: Pulling from wt/b56717475bcc/library/ubuntu
69c262fc30fc: Pull complete
Digest: sha256:e1e96e4b8614acaee0487317cdcf7e8b5b905f00ea8afb88da91b271b0f9b3c0
Status: Downloaded newer image for 334e-84-127-154-163.ngrok-free.app/wt/b56717475bcc/library/ubuntu:latest
334e-84-127-154-163.ngrok-free.app/wt/b56717475bcc/library/ubuntu:latest
(base) munish.chouhan@Munishs-MacBook-Pro ~ % wave --conda-package bwa --wave-endpoint http://localhost:9090
Wave invalid response: [501] {"message":"Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-"}
(base) munish.chouhan@Munishs-MacBook-Pro ~ % wave -i ubuntu --scan-mode required --wave-endpoint http://localhost:9090
Wave invalid response: [501] {"message":"Security scan service is not enabled - Check Wave configuration setting 'wave.scan.enabled' - HTTP status=501 - response=-"}

[pditommaso]

Good or bad?

[munishchouhan]

Good or bad?

Good, I am making one more test with docker compose will report the results soon

[munishchouhan]

tested successfully with docker compose

wave-1  | 11:15:34.089 [main] INFO  i.m.c.DefaultApplicationContext$BootstrapEnvironment - Established active environments: [lite, rate-limit, redis, postgres, prometheus] >> wt=
wave-1  | 11:15:34.640 [main] INFO  i.m.context.DefaultBeanContext - Reading bootstrap environment configuration >> wt=
wave-1  | WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
wave-1  | WARNING: sun.misc.Unsafe::allocateMemory has been called by io.netty.util.internal.PlatformDependent0$2 (file:/app/libs/netty-common-4.1.119.Final.jar)
wave-1  | WARNING: Please consider reporting this to the maintainers of class io.netty.util.internal.PlatformDependent0$2
wave-1  | WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
wave-1  | 11:15:38.219 [main] INFO  i.s.w.metrics.ExecutorsMetricsBinder - + Registering executor metrics binder >> wt=
wave-1  | 11:15:38.433 [main] INFO  io.seqera.wave.redis.RedisFactory - Using redis redis://host.docker.internal:6379 as storage for rate limit - pool minIdle: 0; maxIdle: 10; maxTotal: 50; timeout: 5000 >> wt=
wave-1  | 11:15:39.622 [main] INFO  i.s.w.r.impl.SpillwayRateLimiter - Builds anonymous rate limit: max=10; duration:PT1H >> wt=
wave-1  | 11:15:39.646 [main] INFO  i.s.w.r.impl.SpillwayRateLimiter - Builds auth rate limit: max=10; duration:PT1M >> wt=
wave-1  | 11:15:39.655 [main] INFO  i.s.w.r.impl.SpillwayRateLimiter - Pulls anonymous rate limit: max=250; duration:PT1H >> wt=
wave-1  | 11:15:39.660 [main] INFO  i.s.w.r.impl.SpillwayRateLimiter - Pulls auth rate limit: max=2000; duration:PT1M >> wt=
wave-1  | 11:15:39.668 [main] INFO  i.s.w.r.impl.SpillwayRateLimiter - Timeout errors rate limit: max=100; duration:PT1M >> wt=
wave-1  | 11:15:40.142 [main] INFO  i.s.w.service.pairing.PairingStore - Creating Pairing cache store ― duration=PT720H; lease=PT24H >> wt=
wave-1  | 11:15:40.262 [main] INFO  io.seqera.wave.Bootstrap - Docker.io registry credentials: username=-; password=(null) >> wt=
wave-1  | 11:15:40.265 [main] INFO  io.seqera.wave.Bootstrap - Quay.io   registry credentials: username=-; password=(null) >> wt=
wave-1  | 11:15:40.545 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting... >> wt=
wave-1  | 11:15:41.996 [main] INFO  com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@77f03916 >> wt=
wave-1  | 11:15:42.010 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed. >> wt=
wave-1  | 11:15:43.002 [main] INFO  i.s.w.configuration.HttpClientConfig - Http client config: connectTimeout=PT20S; retryAttempts=5; retryDelay=PT0.5S; retryMaxDelay=null; retryMultiplier=1.75; streamThreshold=65536 >> wt=
wave-1  | 11:15:43.310 [main] INFO  i.s.wave.storage.ManifestCacheStore - Creating Manifests cache store ― duration=PT1H >> wt=
wave-1  | 11:15:43.380 [main] INFO  i.s.wave.auth.RegistryTokenStore - Creating Registry Auth token cache store >> wt=
wave-1  | 11:15:45.493 [main] INFO  io.seqera.wave.proxy.ProxyCache - + Creating Proxy-cache - config=ProxyCacheConfig(duration:PT2M, maxSize:10000, enabled:false) >> wt=
wave-1  | 11:15:47.870 [main] INFO  i.s.wave.filter.RateLimiterFilter - API rate-limiter filter: limitRefreshPeriod=PT1S; limitForPeriod=100; timeoutDuration=PT0.5S; statusCode=429 >> wt=
wave-1  | 11:15:48.079 [main] INFO  i.s.w.s.r.ContainerRequestStoreImpl - Creating Tokens cache store ― duration=PT36H >> wt=
wave-1  | 11:15:49.598 [main] INFO  i.s.w.s.d.s.impl.RedisMessageStream - Creating Redis message stream - consumer=consumer-109498390595394; claim-timeout=PT5S >> wt=
wave-1  | 11:15:49.640 [main] INFO  i.seqera.wave.tower.auth.JwtMonitor - Creating JWT heartbeat - JwtConfig(refreshInterval:PT1H, monitorInterval:PT10S, monitorDelay:PT5S, monitorCount:10, monitorDelayRandomized:PT4.962902758S) >> wt=
wave-1  | 11:15:49.727 [main] INFO  i.seqera.wave.cron.ThreadMonitorCron - Creating check thread monitor cron - dump-interval: PT5M; dump-threshold: 200; dump-file: null; port: 9090 >> wt=
wave-1  | 11:15:49.745 [main] INFO  i.s.w.s.account.AccountServiceImpl - Creating account service (0)
wave-1  |  >> wt=
wave-1  | 11:15:49.915 [main] INFO  i.s.w.controller.ContainerController - Wave server url: http://localhost:9090; allowAnonymous: true; tower-endpoint-url: http://localhost:8000/api; default-build-repo: null; default-cache-repo: null; default-public-repo: null >> wt=
wave-1  | 11:15:50.147 [main] INFO  i.m.l.PropertiesLoggingLevelsConfigurer - Setting log level 'ERROR' for logger: 'com.github.benmanes.caffeine.cache.LocalAsyncCache' >> wt=
wave-1  | 11:15:50.149 [main] INFO  i.m.l.PropertiesLoggingLevelsConfigurer - Setting log level 'ERROR' for logger: 'com.github.benmanes.caffeine.cache.local-async-cache' >> wt=
wave-1  | 11:15:50.150 [main] INFO  i.m.l.PropertiesLoggingLevelsConfigurer - Setting log level 'DEBUG' for logger: 'io.seqera' >> wt=
wave-1  | 11:15:52.891 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 23698ms. Server Running: http://a9318ea9144b:9090 >> wt=
wave-1  | 11:15:56.440 [virtual-executor1] DEBUG i.seqera.wave.http.HttpClientFactory - Creating new neverRedirectsHttpClient: jdk.internal.net.http.HttpClientImpl@2f24626(1) >> wt=
wave-1  | 11:15:56.473 [virtual-executor1] INFO  i.s.w.s.cache.AbstractTieredCache - Cache 'registry-lookup-cache' config - prefix=registry-lookup-cache/v1; max-size: 10000 >> wt=
wave-1  | 11:15:56.508 [virtual-executor1] DEBUG i.seqera.wave.http.HttpClientFactory - Creating new followRedirectsHttpClient: jdk.internal.net.http.HttpClientImpl@e51f262(2) >> wt=
wave-1  | 11:15:58.171 [virtual-executor1] DEBUG i.s.w.a.RegistryCredentialsProviderImpl - Unable to find default credentials for registry 'docker.io' >> wt=
wave-1  | 11:15:58.178 [virtual-executor1] DEBUG i.s.wave.core.RegistryProxyService - Credentials for route path=docker.io/library/ubuntu:latest; identity=PlatformId(user=null, workspaceId=null, accessToken=null, towerEndpoint=null, workflowId=null) => null >> wt=
wave-1  | 11:15:58.987 [virtual-executor1] DEBUG i.s.w.auth.RegistryAuthServiceImpl - Registry auth token for cachekey: 'RegistryAuthServiceImpl$CacheKey(image:library/ubuntu, auth:RegistryAuth(realm:https://auth.docker.io/token, service:registry.docker.io, type:Bearer, refreshable:true, endpoint:https://auth.docker.io/token?service=registry.docker.io), creds:null)' [key-235eac1f7426370b] => eyJ**** >> wt=
wave-1  | 11:15:59.792 [virtual-executor1] DEBUG i.s.w.controller.ContainerController - New container request fulfilled - token=34d4bc65b10b; expiration=2025-06-17T23:15:59.519114179Z; container=docker.io/library/ubuntu:latest; build=null; identity=PlatformId(user=null, workspaceId=null, accessToken=null, towerEndpoint=null, workflowId=null) >> wt=
wave-1  | 11:16:10.593 [virtual-executor4] WARN  io.seqera.wave.ErrorHandler - Build service is not enabled - Check Wave configuration setting 'wave.build.enabled' - HTTP status=501 - response=-
wave-1  | - Request: [POST] /v1alpha2/container
wave-1  | - Headers:
wave-1  |   Content-Length=325
wave-1  |   Host=localhost:9090
wave-1  |   User-Agent=Java-http-client/21.0.4
wave-1  |   Content-Type=application/json >> wt=
wave-1  | 11:16:16.777 [virtual-executor5] WARN  io.seqera.wave.ErrorHandler - Security scan service is not enabled - Check Wave configuration setting 'wave.scan.enabled' - HTTP status=501 - response=-
wave-1  | - Request: [POST] /v1alpha2/container
wave-1  | - Headers:
wave-1  |   Content-Length=187
wave-1  |   Host=localhost:9090
wave-1  |   User-Agent=Java-http-client/21.0.4