minetest
Sky layer API
@sfan5 This is more of a proposal than feature request, if devs are okay with it then I'm willing to begin the work. ~~Possible workaround for #11365~~
Proposal
Introduce a skybox layers/elements API similar in spirit to the current HUD element API, unifying all the currently hardcoded and toggleable elements of the sky. This API would exist in the form of following player-only ObjectRef calls:
handle = ObjectRef:add_sky_layer( SkyLayerSpec )
ObjectRef:replace_sky_layer( handle, SkyLayerSpec )
ObjectRef:remove_sky_layer( handle )
SkyLayerSpec
A 'spec table' with the following fields:
depth = 0
Sorting depth of the sky layer. This determines the rendering order of all the layers.
If nil, the depth is auto-assigned as the highest known element depth plus one.
visual = one of "box" / "sprite" / "stars" / "gradient" / "mesh" / "mesh_center"
The rendering type of the layer.
-
flat: Flat color. This exists mostly for compatibility reasons. -
box: Six sided skybox. -
sprite: Sprite facing the player, akin to a sun or a moon. -
stars: Random noise stars, rendered either as solid points or sprites. -
gradient: One dimensional texture stretched across the sky, using the squashed dot product between a vector and the surface normal as the texture coordinate. -
mesh: Arbitrary 3D model positioned much like a sprite. - `mesh_center' : Arbitrary 3D model sharing the camera's position.
textures = {}
A set of one or more textures required by the visual.
-
boxuses six textures for its sides like the current skybox -
starspicks randomly between the textures -
spriterenders each texture as a layer -
gradientuses the first texture only -
meshandmesh_centerutilize textures just like nodes and objects do
color = { r=1, g=1, b=1, a=1 }
An RGBA color multiplying the whole layer's color. Modify alpha to control opacity.
color = { [0] = { r=1, g=1, b=1, a=1 }, [0.5] = { r=0.1, g=0.1, b=0.1, a=1 } }
Alternative form of color, which evaluates an animation curve where t = normalized orbital time. Use for day/night fading, sunsets, sunrises, etc.
gradient_vector = { x=0, y=1, z=0 }
A vector for the gradient visual to compute the dot product against. Up by default.
light = { r=1, g=1, b=1 }
light_shadow = true
A colored light associated with the layer. This is the replacement for the current dynamic sunlight and moonlight used for shading and shadows. The direction of the light is always such that it appears to come from where a sprite or mesh would be present.
If light_shadow is true, the light will cast a shadow. The brightest light above the horizon will be used as the current shadow caster.
angle = { x=0, y=0, z=0 }
A euler angles vector specifying the initial rotational position of the layer.
For sprite and mesh this controls the actual position, which is +Z by default.
Stars, gradient, box and centered mesh are all
mesh_rotation = { x=0, y=0, z=0 }
mesh_rotation_space = one of "local"/ "world"
A euler angles vector specifying the local rotation of the mesh visual. Ignored otherwise.
The space enum determines whether the rotation is absolute, or an offset from the camera-facing orientation.
orbit_axis = { x=0, y=0, z=1 }
orbital_period = 0
orbital_offset = 0
Orbital parameters. Orbital axis determines the axis around which the object rotates as time progresses, orbital period determines how many revolutions it makes per server day, and orbital offset is a normalized (0-1) offset applied to the orbit.
Set orbit_axis to nil to disable orbiting.
blending = one of "opaque" / "alpha_test" / "alpha_blend" / "additive" / "multiply"
Transparency and blending mode. All are achievable with basic OpenGL functionality and do not require shaders.
Rendering and compositing
The sky is rendered after the world. The stencil buffer is first used to mark all the fragments available to the skybox, and a stencil test is used in subsequent passes. Each layer is treated as being infinitely away from the camera, and rendered in isolation with Z testing and writing enabled, but subsequent layers always overwrite the previous layer regardless of the depth, which is achieved by clearing the depth between layers. Layers are unlit, they utilize vertex colors if present, and may be blended using various modes.
Compatibility and transition
The current sky elements are assigned hardcoded negative indices, which are never assigned to user elements. APIs controlling them are deprecated but still work, by emulating calls to the new API. Fog is currently an unsolved problem, where do we move it? And what additional features, if any, are required to support the more subtle features of the current default sky?
Alternative transition plan
Develop the API alongside the current one and instead of emulating old skyboxes, require setting an "extended" sky type before these fancy layers can be used. This requires no emulation, but feels dirty.
Optional features
gradient, mesh and mesh_center as well as advanced blending modes are not necessary to emulate the current behavior and may be skipped in an initial implementation.
Would this allow a 3-layers cloud system like that? https://www.youtube.com/watch?v=swfWAO7H_lA
As a proof of concept, yes I suppose; add three "box" layers each with a different cloudy texture and give them different orbital periods. A dedicated cloud visual could also be implemented, but clouds are tricky - currently you can reach and touch the minecraft clouds as they are rendered in world space. So either do a different api for clouds, or give cloud layers special treatment. Separate API is my preference.
That's two, anyone else? I've found a bit of a technical snag regarding toggling lights; it seems that we do not have any support for shader variants. Shaders are apparently compiled once in client init, and further used with the assumption that the underlying GL program object can not change. A refactor of shaders and materials might be in order.
Sounds good generally.
When reading this I had a few (probably trivial) questions:
- It doesn't make sense to define more than one layer with
flatorbox, does it? (transparency?) - How does "sharing the camera's position" work?
- "Transparency and blending mode": Honestly no idea if Irrlicht exposes additive and multiply
The current sky elements are assigned hardcoded negative indices, which are never assigned to user elements.
If possible I think it'd be nicer if the current sky could be added as default layers in the API. Then if a game want something else it could just clear all layers and add their own.
And what additional features, if any, are required to support the more subtle features of the current default sky?
Good question. Just browsing through the code I don't think you could represent e.g. this in any way: https://github.com/minetest/minetest/blob/2db6b07de1e45c56f6135363939dbb3781336514/src/client/sky.cpp#L167-L179
Optional features [...]
If you are willing to work on this I think it'd be a good approach to first implement the features needed to move the current default sky "into" the layer API. Then when that works the rest of the features should be added.
It doesn't make sense to define more than one layer with flat or box, does it? (transparency?)
The "flat" visual is just there for compatibilty, it would at the very least be used internally to simulate the single color sky mode. But not exposing it is actually more work than leaving it there, it doesn't really hurt to have a flat color layer. So it's there for completeness. (and sometimes all you want is a color overlay, this allows doing that without hacks like a 1x1 white texture box)
And yes, since every layer has transparency, several boxes may be used. They may rotate at different speeds and orbits in a dynamic way. You can use that to do fancy layered clouds or even some weird psychedelic background in some sort of nether world. (Vertex colored spheres are also viable for these purposes.)
How does "sharing the camera's position" work?
By sharing the camera's position I mean that every sky element is treated as if it were at an infinite distance. That implies that there is no parallax as the camera moves; effectively the position column of the view matrix is always zero, and the camera is always in the middle of the box. Just like with the current skybox, really, it's simply a more formal way of stating that.
The actual scale of the sky elements is irrelevant in this case because we should be using stencil and depth operations to always place them behind the world and sort them relative to each other.
Also, unlike the current built in skybox, the sky should be rendered after the world - this saves GPU fill, the current skybox is doing it wrong by being first. It may not be much but it'll help those poor software renderers like Mesa.
"Transparency and blending mode": Honestly no idea if Irrlicht exposes additive and multiply
I'm just going off of what I know is possible in OpenGL 1.0 without any shaders. I assume Irrlicht allows you to just set the GL blend modes in some way, if not, I'll do what is possible.
If possible I think it'd be nicer if the current sky could be added as default layers in the API.
That's what the first proposition is, give them hardcoded negative IDs and let the deprecated API work by manipulating those default elements behind the scenes, while also letting new modders edit them raw or scrap them and build a sky from a clean slate. Old packets for sky manipulation would no longer be sent, though they could still be interpreted by the client to allow playing servers using the older protocol.
Just browsing through the code I don't think you could represent e.g. this in any way: [...]
Look at the second representation of color, in the animation curve form. Wouldn't that do it?
Sounds fantastic! Mesh should support animations too, as well as obit
So we could have like monolithic creatures or little spaceships flying around in the sky as the planet slowly turns
Mesh should support animations too, as well as obit
The plan is to support orbits on every visual type except flat. Skinned meshes sound interesting but certainly not as a priority. They would probably simply play a specified animation range.
Look at the second representation of color, in the animation curve form. Wouldn't that do it?
Hm I could see that working.
Though the colors defined there don't directly depend on the normalized orbital time (as proposed) but instead on m_time_brightness and m_brightness, the latter of which is also somehow depends on the ingame situation (src/client/game.cpp#L3702-L3719)
I guess there's two possible parameters that can be used as (t), first one is the actual time, the second one is the day/night override. Besides that, is ignoring a small detail like this an acceptable concession, if the goal is to encourage custom sky elements anyway?
I was brought to this discussion after asking on Discord and nearly opening my own duplicate issue. What I was looking for is a way to use a skybox cubemap with transparency that also allows the sky to be seen behind it: I was saddened that even now, this much isn't yet supported.
On the API page the set_sky function indeed lists only 3 modes for the type parameter: Regular, skybox, color. The first two can't be used together, you aren't allowed to have a cubemap covering just parts of the sky but letting the sun / moon / stars shine through, nor multiple cubemaps you can layer on top of each other and perhaps rotate independently. If that much isn't too difficult to implement, I really hope at least this can be implemented soon.
Beyond transparent skyboxes, it would be nice if other elements of this API can be supported, though I find that the most urgent. Being able to use your own stars would allow games and mods to do some beautiful things, with moving sprites attached to the sky we could also allow custom planets and other large structures!
