configure velocity proxy using env variable

[alexk201]

The ability to configure the Velocity forwarding secret via the environment variable PAPER_VELOCITY_SECRET was introduced in #10127. However, since Velocity support is disabled by default, using this feature still requires manual modification of paper-global.yml.

To enable full automation of Velocity configuration via environment variables, this PR adds support for two additional env vars:

• PAPER_VELOCITY_ENABLED - enables the Velocity proxy support (true or false)
• PAPER_VELOCITY_ONLINE_MODE - sets the online-mode value (true or false)

This ensures that all necessary Velocity settings can be configured via environment variables without manual intervention.

Note: paper-global.yml is currently not generated when using the --initSettings CLI flag, which limits the ability to preconfigure the server without a full startup. Unless that behavior changes, environment variables remain the only viable option for automating this configuration in containerized environments.

[lynxplay]

The reason the environment variable for the secret was exposed was secret management in k8, especially when also applied in a multi proxy setting.

A "configure your server via environment variables instead of yml" was not really the point of the original PR so I am a bit hesitant on this PR right now.

I'd be semi down to investigate the generation of config files with initSettings, tho our config setting requires some data that is probably not available at that time of the servers startup.

[alexk201]

I understand that the focus of the original PR was secret management. As mentioned in my PR description, the secret alone does not suffice to configure velocity though. And I am really wondering how other people managed to get paper running on k8s without the newly introduced environment variables. K8s ConfigMaps don't work as they are mounted read only. The first thing spigot and paper do at startup is reading and then re-writing the yaml config to disk so it crashes. Two options remain: Env variables or a persistent volume that behaves as a regular disk. The first option is contained in this PR. The second option would be ok-ish too but it's sooo hacky: If you have servers that get discarded anyways (like minigame servers), you don't want to use a persistent volume. If you use a temporary filesystem instead, you first have to create the required configuration files which isn't possible using the initSettings command as mentioned before. IMHO, env variables are the best option for configuring any software running in a containerized environment so I'd love to see it in Paper as well.

[alexk201]

Coming from the Java Enterprise world (Application Servers, Quarkus, and Spring Boot), I might be a bit spoiled - we've had the Eclipse MicroProfile for several years now. One of its standout features is the MicroProfile Config API, which provides a flexible abstraction layer over various configuration sources. Most implementations support properties files, YAML, environment variables, and even more enterprise-specific options like JDBC, JNDI, and HashiCorp Vault - though many of those might not be particularly relevant for Paper.

I'm not sure how many people (besides me) are running Paper in Kubernetes environments, but it would be amazing to make everything configurable from any source. I'd be happy to contribute some time and share my experience from the Java Enterprise ecosystem to explore whether Eclipse MicroProfile (or a similar abstraction) could make sense for Paper.

That said, I'd love to hear the team's thoughts on this first. From what you mentioned, it seems like config files in Paper have some nuanced lifecycle dependencies - for example:

I'd be semi down to investigate the generation of config files with initSettings, tho our config setting requires some data that is probably not available at that time of the server's startup.

So before diving deeper, does this idea align at all with where Paper wants to go with configuration? Or is this a case of "nice in theory, but too heavy for this use case"?

[electronicboy]

env-based config stuff has been on the "maybe cool to have", but it would need to be integrated in some manner into the configurate configuration library. However, such a concept just seems fairly limited, especially when it comes to non-primitive entries and such. We generally won't be migrating away from configurate unless something solves something it doesn't in a substantial manner, there just isn't the demand or resources for us to cater to these sorta setups, anybody at the scale of needing such envs is generally forking to cater for their needs anyways

[Machine-Maker]

I've thought about setting up something where simple primitives could be overriden by environment variables. I don't want to get crazy with it, where we have some crazy way env vars can set lists or full objects. But detecting simple enough primitive values, and generating an env var key based on its path should be doable.

Have to do it in such a way to merge it into the config node without writing it back out I think, that might be harder.

[lynxplay]

An alternative we could potentially investigate a bit further is supporting ro configuration files. In general, our migrations and defaults don't need to be written back to file, just obviously makes editing the files worse if keys are missing and the file is not on the latest version but that can be up to the administrators then.

I believe we currently don't fail if the paper configs cannot be written to? We presumably error very loudly but it should startup and run the migrations on the in-memory representation. Idk if that is applicable to spigot.yml and bukkit.yml, those use a different system rn.

[electronicboy]

RO config files are one of those weird ones where I think we shifted some concessions towards (but, this is one of the areas why I kinda supported it, even if it makes no sense for most setups and is often an early warning sign something is wrong)

afaik, k8s would let you expose a config file in the container from the ConfigMap or whatever it was, but I don't have the resources to waste time learning all of that stuff yet

[alexk201]

Unfortunately, paper does not ignore read only (in a k8s environment)

[19:11:54 ERROR]: Encountered an unexpected exception
org.spongepowered.configurate.ConfigurateException: []: java.nio.file.FileSystemException: /opt/paper/config/.7199899733415571904970619paper-global.yml.tmp -> /opt/paper/config/paper-global.yml: Device or resource busy
        at org.spongepowered.configurate.loader.AbstractConfigurationLoader.save(AbstractConfigurationLoader.java:211) ~[configurate-core-4.2.0-20250225.064233-204.jar:?]
        at io.papermc.paper.configuration.Configurations.trySaveFileNode(Configurations.java:113) ~[paper-1.21.7.jar:1.21.7-DEV-68488bd]

The server crashes and shuts down. Maybe it's overkill to migrate the entire file configuration to a different system. Everything that can be configured at runtime like game difficulty doesn't need to be exposed using some other configuration system. Properties that need to be set before server startup would be nice to be exposed somewhere else.

[alexk201]

In case anyone wants to reproduce it in k8s, here's a shortened version of my setup:

apiVersion: v1
kind: ConfigMap
metadata:
  name: paper-global-config
data:
  paper-global.yml: |
    _version: 29
    anticheat:
      [...]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: paper
  labels:
    app.kubernetes.io/name: paper
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: paper
  template:
    metadata:
      labels:
        app.kubernetes.io/name: paper
    spec:
      containers:
        - name: paper
          image: my-registry/paper:1.21.7-26
          volumeMounts:
            - name: config-volume
              mountPath: /opt/paper/config/paper-global.yml
              subPath: paper-global.yml
      volumes:
        - name: config-volume
          configMap:
            name: paper-global-config

[lynxplay]

From a glance at the offending code, we only support proper denied writes, e.g. missing write permissions for the server user. You could hence probably fix this by defining the files as read-only via defaultMode: 0444 in your volumes[0].configMap.

With semi obvious note that mounting in a single file is a bit of a hack so you might also need to define the world defaults. This makes this technique kinda questionable for the spigot and bukkit config but yea. That should throw you an agressive warning about not being able to write but it should correctly let you start the server.

[alexk201]

Thanks for investigating on the topic of Kubernetes ConfigMaps even though this is not your main concern :) Unfortunately, the defaultMode doesn't prevent the error. After adding it, the permissions are looking good:

$ kubectl exec <my-pod> -- ls -la config
total 4
drwxr-xr-x. 2 root root   30 Jul 29 09:27 .
drwxr-xr-x. 1 root root   32 Jul 29 09:27 ..
-r--r--r--. 1 root root 3700 Jul 29 09:25 paper-global.yml

This still results in the same stack trace. Problem is that the underlying exception is a java.nio.file.FileSystemException instead of a java.nio.file.AccessDeniedException.

I should've added the complete stack trace earlier...

[09:27:51 ERROR]: Encountered an unexpected exception
org.spongepowered.configurate.ConfigurateException: []: java.nio.file.FileSystemException: /opt/paper/config/.2390207891523211899288440paper-global.yml.tmp -> /opt/paper/config/paper-global.yml: Device or resource busy
	at org.spongepowered.configurate.loader.AbstractConfigurationLoader.save(AbstractConfigurationLoader.java:211) ~[configurate-core-4.2.0.jar:?]
	at io.papermc.paper.configuration.Configurations.trySaveFileNode(Configurations.java:113) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at io.papermc.paper.configuration.Configurations.initializeGlobalConfiguration(Configurations.java:138) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at io.papermc.paper.configuration.Configurations.initializeGlobalConfiguration(Configurations.java:108) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at io.papermc.paper.configuration.PaperConfigurations.initializeGlobalConfiguration(PaperConfigurations.java:225) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at net.minecraft.server.dedicated.DedicatedServer.initServer(DedicatedServer.java:171) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at net.minecraft.server.MinecraftServer.runServer(MinecraftServer.java:1164) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at net.minecraft.server.MinecraftServer.lambda$spin$2(MinecraftServer.java:310) ~[paper-1.21.8.jar:1.21.8-DEV-e1f2189]
	at java.base/java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.nio.file.FileSystemException: /opt/paper/config/.2390207891523211899288440paper-global.yml.tmp -> /opt/paper/config/paper-global.yml: Device or resource busy
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[?:?]
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
	at java.base/sun.nio.fs.UnixFileSystem.move(Unknown Source) ~[?:?]
	at java.base/sun.nio.fs.UnixFileSystemProvider.move(Unknown Source) ~[?:?]
	at java.base/java.nio.file.Files.move(Unknown Source) ~[?:?]
	at org.spongepowered.configurate.loader.AtomicFiles$AtomicFileWriter.close(AtomicFiles.java:121) ~[configurate-core-4.2.0.jar:?]
	at java.base/java.io.BufferedWriter.implClose(Unknown Source) ~[?:?]
	at java.base/java.io.BufferedWriter.close(Unknown Source) ~[?:?]
	at org.spongepowered.configurate.loader.AbstractConfigurationLoader.save(AbstractConfigurationLoader.java:208) ~[configurate-core-4.2.0.jar:?]
	... 8 more

[lynxplay]

Interesting, I did validate my claim at least on a local kind cluster before replying o.O but yea, permissions look good. An alternative we could consider, that might be a lot easier for us to implement, is allowing definition of overrides to the config yaml files via secondary filesets. I am thinking java -jar server.jar --paper.global.values=/some/path/not/in/server/folder which can then directly be pointed at a config map mount point.

Limiting the overriding to yaml files would solve the "how do I define a map of lists of maps in env variables" problem by simply ignoring it and should be easily implemented via the existing merge functionalities in configurate. Those files could then 100% be read only with the server not even attempting a write-back for migration and simply ignoring outdated keys during the merge.

//edit: On the note of :ro layering of configs, there is another usecase such a solution could be beneficial for. I'll look for feedback from the broader team on this.

[alexk201]

I am not 100% sure but the difference in behaviors might also be caused by the JREs we're using. File system handling might be a little different depending on the implementation but I'm just speculating... Anyways I like your proposed solution especially since merge sounds like I can just specify the keys I am interested in instead of the entire yaml file?

[lynxplay]

Yea exactly, think helm chart value merging, at least that would be my vision for this.

[alexk201]

Sounds good! The merge file path (you called it --paper.global.values in your example) will be configurable using ENV variables, right? :rofl:

[Funasitien]

This would help a lot for local devlopment! I do use kubes too, and this would save me so much time!