c-mera
c-mera copied to clipboard
kiselgra
Document how to create GLSL shader
I have a hard time figuring out how to declare a GLSL compute shader:
I want the code to expand into this:
#version 430 core
layout (local_size_x=32, local_size_y=32) in;
layout (binding=0,r32f) uniform image2D data;
layout (binding=2,r32ui) uniform uimage2D lut;
const int imw=380, imh=241;
const int ng=256;
void main(void)
{
All I came up with up to now is this:
(simple-print
(progn
(comment (format nil "#version 430 core~%") :prefix "")
(layout (local-size-x 32) (local-size-y 32) ) (comment "in;" :prefix "")
(layout (binding 0) (nil 'r32f))
(function main () -> void )))
I don't know how to make the r32f qualifier work and it is not clear to me how to declare the images or swizzle access like this:
texture(samp2,texcoord.xy).r
Hi there,
I would suggest following approach for the compute shader:
(simple-print
(progn
(comment (format nil "#version 430 core~%") :prefix "")
(decl (((layout (local-size-x 32) (local-size-y 32)) in nil nil)
((layout (binding 0) (r32f)) uniform image2D data)
((layout (binding 2) (r32ui)) uniform uimage2D lut)
(const int imw 380)
(const int imh 241)
(const int ng 256))
(function main () -> void
(return)))))
Im sorry, but this line (decl (((layout (local-size-x 32) (local-size-y 32)) in nil nil) is currently not working without the nils (a declaration demands a type and a name but in is only a qualifier).
And for the siwizzling try:
(oref (texture samp2 texcoord.xy) r)
Our reader cannot handle swizzling with function calls.
You can evade that with an explicit oref.
I think we should consider adding a few convenience macros to glslgen. Maybe something like that:
(defmacro glsl-version (v)
`(comment (format nil "#version ~a~%" ,v) :prefix ""))
and
(defmacro local-size (x y)
`(decl (((layout (local-size-x ,x) (local-size-y ,y)) in nil nil))))
Then we could write the code in a less painful way:
(progn
(glsl-version 430)
(local-size 32 32)
(decl (((layout (binding 0) (r32f)) uniform image2D data)
((layout (binding 2) (r32ui)) uniform uimage2D lut)
(const int imw 380)
(const int imh 241)
(const int ng 256))
(function main () -> void
(return))))
I'd also like to make it more clear that we are expressing a shader with this block, maybe with something like this:
(defmacro define-compute-shader ((&key (v "430") (x 16) (y 16)) &body body)
`(progn
(glsl-version ,v)
(local-size ,x ,y)
,@body))
resulting in code as the following:
(define-compute-shader (:v "430 core" :x 32 :y 32)
(decl (((layout (binding 0) (r32f)) uniform image2D data)
((layout (binding 2) (r32ui)) uniform uimage2D lut)
(const int imw 380)
(const int imh 241)
(const int ng 256))
(function main () -> void
(return))))
@plops: What do you think about adding something along those lines to glslgen? Do you have other suggestions?
@lispbub: What do you think?
@lispbub: I still think that the local-size construct results in ugly code, maybe we should consider having a special node for that in our tree, to be able to get a nicer, more standard formatting. Also, with the r32f specifiers there is a superflous trailing space, can we get rid of that?
I'm quite happy with @lispbub 's suggestions. Here is a (useless) example of how my shaders look like now:
(let* ((w 32)
(h 32))
(simple-print
(progn
(comment (format nil "#version 430 core~%") :prefix "")
(decl (((layout (local_size_x w) (local_size_y 32)) in nil nil)
((layout (binding 0) (r32f)) uniform image2D data)
((layout (binding 2) (r32ui)) uniform uimage2D lut))
(function main () -> void
(decl ((ivec2 pos (funcall ivec2 gl_GlobalInvocationID.xy))
(float texel (oref (funcall imageLoad data pos) r))
(int g (funcall int (funcall floor (* *ng* texel))))
(float x (* .3 (- pos.x w))))
(progn ;; if (and (< i imw) (< j imh))
(if (and (<= 12 pos.x) (< pos.x 321))
(funcall imageStore data pos (funcall vec4
(* (cl:/ 1s0 (cl:* w h))
(funcall mix
(oref (funcall imageLoad lut (funcall ivec2 pos.x g)) r)
(oref (funcall imageLoad lut (funcall ivec2 (+ pos.x 1) g)) r) x))
0 0 0))))))))))
It expands into:
#version 430 core
layout(local_size_x = 32, local_size_y = 32) in ;
layout(binding = 0, r32f ) uniform image2D data;
layout(binding = 2, r32ui ) uniform uimage2D lut;
void main(void)
{
ivec2 pos = ivec2(gl_GlobalInvocationID.xy);
float texel = imageLoad(data, pos).r;
int g = int(floor(256 * texel));
float x = 3.00000000e-1 * (pos.x - 32);
if ((12 <= pos.x) && (pos.x < 321)) {
imageStore(data, pos, vec4(9.76562500e-4 * mix(imageLoad(lut, ivec2(pos.x, g)).r, imageLoad(lut, ivec2(pos.x + 1, g)).r, x), 0, 0, 0));
}
}
Note that I had to apply my patch for https://github.com/kiselgra/c-mera/issues/20 for the floating point number to be printed with sufficient precision.
I'm not sure if it is necessary to introduce macros like local-size or glsl-version. The ones given above seem to be to primitive (local-size can have 3 dimensions, glsl-version can have additional arguments).
I think it would be more helpful to collect example code. If the macros were in the c-mera repo, I could certainly learn from them. So in that way, they would help.
One issue that hasn't been addressed is the shared qualifier. For now I just call (cgen::add-qualifier 'shared). I think there are more qualifiers like this (coherent, ...).
I have another suggestion: I read on https://www.opengl.org/wiki/Core_Language_%28GLSL%29#Compilation_settings that GLSL doesn't allow variable declarations inside an if expression. Would that be easy to catch in c-mera? But I doubt the advantage of such a check outweighs the maintainability decrease in c-mera.
Yes, the suggested code was just an example, if we add something along those lines then we should aim for it to be generally applicable :)
Adding the qualifiers should be easy, we'll add them. If you have a reference available, can you put up a link? Otherwise I'll just check the spec when I get to it.
Regarding the proposed check, I think this would be possible, but I'm not sure it would be worth the trouble. Usually c-mera deferes some part of the error handling to the underlying compiler. If we decide to spend more work on handling that case it would only be consistent to move more error handling to c-mera in general. So for now I'd defer that point.
I'll also add some details to the documentation in the following days.
I usually look at the reference card for OpenGL 4.3 because that is the first version with compute shader: https://www.khronos.org/files/opengl43-quick-reference-card.pdf
Page 7 lists qualifiers and built-in variables.
As an example here are the variables that I have accumulated while working with OpenGL for a while. Not many of them are for GLSL, though.
(use-variables NULL stderr GL_ARRAY_BUFFER GL_TRUE GL_FALSE gl_GlobalInvocationID ogl_LOAD_FAILED
GLFW_SAMPLES GLFW_CONTEXT_VERSION_MAJOR GLFW_CONTEXT_VERSION_MINOR GL_TIME_ELAPSED
GL_QUERY_RESULT_AVAILABLE GL_QUERY_RESULT GL_MAX_COMPUTE_SHARED_MEMORY_SIZE STRINGIZE __LINE__
GLFW_OPENGL_PROFILE GLFW_OPENGL_CORE_PROFILE GLFW_PRESS GLFW_KEY_ESCAPE GL_PROGRAM_BINARY_LENGTH
GL_TRIANGLE_FAN GL_COLOR_BUFFER_BIT GL_FLOAT GL_TRIANGLE_FAN GL_UNSIGNED_SHORT GL_FLOAT
GL_UNSIGNED_INT GL_FRAGMENT_SHADER GL_VERTEX_SHADER GL_COMPUTE_SHADER GL_LINK_STATUS GL_COMPILE_STATUS
GL_VALIDATE_STATUS GL_COMPUTE_SHADER GL_RED GL_RED_INTEGER GL_R8UI GL_R8 GL_R32F GL_R32UI
GL_UNSIGNED_BYTE GL_CLAMP_TO_EDGE GL_TEXTURE_WRAP_T GL_TEXTURE_WRAP_S GL_TEXTURE_2D
GL_TEXTURE_MIN_FILTER GL_NEAREST GL_TEXTURE_MAG_FILTER O_RDONLY O_WRONLY O_RDWR O_CREAT SEEK_SET
GL_TEXTURE0 GL_TEXTURE1 GL_TEXTURE2 GL_READ_WRITE GL_STATIC_DRAW GL_ARRAY_BUFFER GL_NO_ERROR
GL_INVALID_ENUM GL_INVALID_VALUE GL_INVALID_OPERATION GL_INVALID_FRAMEBUFFER_OPERATION
GL_OUT_OF_MEMORY GL_ELEMENT_ARRAY_BUFFER GL_SHADER_IMAGE_ACCESS_BARRIER_BIT
gl_LocalInvocationID gl_WorkGroupID)
I can't figure out how to emit a shader layout line, where the type information for the image (e.g. r32f or r32ui) can be given in a variable. I want to generate code that can easily be switched from 2D to 3D arrays and between float and uint32 representation.
(defclass shader ()
((global-x :accessor global-x :initarg :global-x :type fixnum)
(global-y :accessor global-y :initarg :global-y :type fixnum)
(global-z :accessor global-z :initarg :global-z :type fixnum)
(source :accessor source :initarg :source :type string)))
(defun make-shader (&key (x 1) (y 1) (z 1) (source ""))
(make-instance 'shader :global-x x :global-y y :global-z z :source source))
(let* ((w 4) (h 3) (d 2)
(internal-format GL_R32F) (format GL_RED) (shader-format 'r32f) (shader-uni #\Space) (c-format GL_FLOAT) (c-format-sym "f") (c-format-type 'float)
;(internal-format GL_R32UI) (format GL_RED_INTEGER) (shader-format 'r32ui) (shader-uni 'u) (c-format GL_UNSIGNED_INT) (c-format-sym "d") (c-format-type 'uint32_t)
)
(defparameter *cs-clear-xd*
(make-shader :x 1 :y h :z d :source
(with-output-to-string (*standard-output*)
(simple-print
(progn
(comment (format nil "#version 430 core~%") :prefix "")
(decl (((layout (local_size_x w) (local_size_y 1) (local_size_z 1)) in nil nil)
((layout (binding 0) (shader-format)) uniform (intern (format nil "~aimage~dD" shader-uni (cl:if (cl:= d 1) 2 3))) uhist))
(function main () -> void
(if (cl:if (cl:= d 1)
(and (< gl_GlobalInvocationID.x w) (< gl_GlobalInvocationID.y h))
(and (< gl_GlobalInvocationID.x w) (< gl_GlobalInvocationID.y h) (< gl_GlobalInvocationID.z d)))
(funcall imageStore uhist
(cl:if (cl:= d 1)
(funcall ivec2 gl_GlobalInvocationID.xy)
(funcall ivec3 gl_GlobalInvocationID.xyz))
(funcall (intern (format nil "~avec4" shader-uni))
(cl:if (cl:= d 1)
(+ gl_GlobalInvocationID.x (* 100 (+ gl_GlobalInvocationID.y)))
(+ gl_GlobalInvocationID.x (* 100 (+ gl_GlobalInvocationID.y (* 100 gl_GlobalInvocationID.z)))))
)))))))))))
I would expect shader-format to expand into r32f. But that doesn't work.
Hi,
in this case you'll have to substitute your values before the code is evaluated. Otherwise the symbols in the declaration will be used verbatim as identifiers.
Trying to stay close to what you have written I would change the make-shader function to be handed a tree, not a string, and evaluate it as a c-mera expression:
(defun make-shader (&key (x 1) (y 1) (z 1) source)
(make-instance 'shader
:global-x x :global-y y :global-z z
:source (with-output-to-string (*standard-output*)
(simple-print
(eval source)))))
I have already moved some of the output-handling stuff into this function.
Then you can use quotation to get the shader-format filled in:
(let ((w 4) (h 3) (d 2)
(internal-format GL_R32F)
(format GL_RED)
(shader-format 'r32f)
(shader-uni #\Space)
(c-format GL_FLOAT)
(c-format-sym "f")
(c-format-type 'float))
(defparameter *x*
(make-shader :x 1 :y h :z d
:source `(progn
(comment (format nil "#version 430 core~%") :prefix "")
(decl (((layout (local_size_x ,w) (local_size_y 1) (local_size_z 1)) in nil nil)
((layout (binding 0) (,shader-format)) uniform (intern (format nil "~aimage~dD" ,shader-uni ,(cl:if (cl:= d 1) 2 3))) uhist))
(function main () -> void
(return)))))))
This is just the first thing that came to mind, hope it helps! :)
Hi,
I just added some code to adapt the gl.xml specification. All enums should now be available, both via, e.g., GL_TEXTURE_2D and (I think more conveniently, and similar to cl-opengl) via gl:texture-2d.
Hope this doesn't break anything for you.