[Question] Questions about preset authoring

[AntiSol]

Please confirm the following points:

• [x] This question is NOT about the Android apps in the Play Store
• [x] I have searched the project page to check if the question was already asked elsewhere

Topic

General Request

Your Request

Hello,

I tried to join the discord to ask this, but it said "unable to accept invite", providing no other error message or information of any kind. Not sure what I can do about that 🤷:

I have a few questions about preset authoring.

0. Best source for info?

I've been reading the Preset Authoring guide on the wiki (and also the old original version), but there seems to be quite a bit of info which is referring to milkdrop rather than projectM. For instance the guide says that if I specify a that a shape should be textured with shapecode_x_textured but don't provide an ImageURL, then it should be textured with the previous frame, but this doesn't seem to work as advertised. I assume because projectM hasn't implemented that feature?

What is the best source of info WRT authoring presets specifically for projectM? Is there some better source than the wiki on github?

1. Possible to maintain aspect ratio of textures?

I've been playing around with creating a projectM preset to draw an image, with code like:

shapecode_7_enabled=1
shapecode_7_sides=4
shapecode_7_textured=1
shapecode_7_ImageURL=my_image.tga
shapecode_7_x=0.5
shapecode_7_y=0.5
shapecode_7_rad=1.5
shapecode_7_tex_zoom=0.6
shapecode_7_r=1
shapecode_7_g=1
shapecode_7_b=1
shapecode_7_a=1
shapecode_7_r2=1
shapecode_7_g2=1
shapecode_7_b2=1
shapecode_7_a2=1

My image is square - 300x300. The problem I'm having is that when I run my preset in a widescreen resolution like 1280x720, the image is stretched. I have the 'aspect ratio correction' option enabled. If I remove the ImageURL from the shape, it shows up as a square, even at 1280x720. It looks like the texture is being stretched (but not resized?) to match the window resolution?

2. Can custom shapes be arbitrarily defined / textured?

I see I can define regular polygons by setting the number of sides, which gives me evenly-sided shapes. What I'd like though is to have a shape where I define the x/y coordinates of each point. In particular I'd like to have a trapezoid or parallelogram, like so:

I'd also like to be able to have my texture image deformed to this shape, i.e this would allow me to make it look like my image has perspective, and to shift that "perspective" over time.

I can achieve something similar to what I want to do by applying a perspective transform to the image before I save it, setting up multiple shapes for the different perspectives I want, and toggling their 'enabled' state in a per_frame statement, but this doesn't allow for smooth transforms. e.g I have 'left' and 'right' perspectives which i can switch between, but there's no way to have it smoothly transform from one to the other, which is what I'd like.

texture_left.tga:

texture_right.tga:

per_frame_1=q30=sin(time * 2);

shapecode_1_enabled=0
shapecode_1_sides=4
shapecode_1_textured=1
shapecode_1_ImageURL=texture_left.tga
shapecode_1_x=0.5
shapecode_1_y=0.5
shapecode_1_rad=1.5
shapecode_1_tex_zoom=0.6
shapecode_1_r=1
shapecode_1_g=1
shapecode_1_b=1
shapecode_1_a=1
shapecode_1_r2=1
shapecode_1_g2=1
shapecode_1_b2=1
shapecode_1_a2=1
shape_1_per_frame6=enabled=if(above(q30,0),1,0); // visible when q30 > 0

shapecode_1_enabled=0
shapecode_1_sides=4
shapecode_1_textured=1
shapecode_1_ImageURL=texture_right.tga
shapecode_1_x=0.5
shapecode_1_y=0.5
shapecode_1_rad=1.5
shapecode_1_tex_zoom=0.6
shapecode_1_r=1
shapecode_1_g=1
shapecode_1_b=1
shapecode_1_a=1
shapecode_1_r2=1
shapecode_1_g2=1
shapecode_1_b2=1
shapecode_1_a2=1
shape_1_per_frame6=enabled=if(above(q30,0),0,1); // visible when q30 < 0

but it would be much better if I could supply 4 x/y coordinates to define the corners of my shape, and have my texture stretched to fit that shape. That way I could have just one texture file and smoothly move from "left" to "right". Is there any way to accomplish what I want?

I have about a thousand other questions, but those are the most pressing for now 😄

[revmischa]

Not sure what's up with the discord invite link but I just made a new one: https://discord.gg/N9DyQfCH4j

[AntiSol]

Not sure what's up with the discord invite link but I just made a new one: https://discord.gg/N9DyQfCH4j

same deal: Whoops... Unable To Accept Invite

[kblaschke]

Can't say much about the Discord error, but to answer the actual question about shapes:

The textured option only enables drawing the current warped preset image (e.g. the output of the "warp" shader) into the shape itself. Besides the zoom, there's no other way of changing the texture coordinates within the shape.

The shape itself is always drawn as a so-called "triangle fan", which means it has a center vertex, and then 3-100 outer vertices which are drawn in a circle around that center point, using rad as the distance. The ang option defines at which angle the first vertex is drawn, and thus also controls the rotation of the texture.

projectM (and only projectM) supports an additional option to place an image onto the shape, which was used to display the M/headphones logo in the "idle" preset. This features was quickly hacked in there, and really only meant as an internal way to improve this single preset, not for general use. The texture only displays properly if it is square and you set the texture zoom to sqrt(2) / 2 which is 0.707106781187 and the number of sides to 4, otherwise it'll either repeat or be cut off.

Aspect ratio correction in custom shapes is sadly broken beyond repair, as "fixing" it would break hundreds of presets which carefully align shapes and custom waves. Milkdrop's original code explicitly says "DON'T TOUCH!" in the respective lines for that exact reason.

If you want to make a preset that also works in Milkdrop and its numerous forks, the best and probably only compatible way to render textures and make them look 3D-ish is inside the "comp" shader and use raycasting. This would also enable you to use a single, flat texture. There are plenty of tutorials out there on how to accomplish this, e.g. look up "Art of Code" on YouTube. Most do it using GLSL, but the code should be relatively easy to translate to HLSL. Here's an example preset which renders a raycasted sphere, without textures, but might be a good starting point.

[AntiSol]

Thanks for your excellent and thorough response! This confirms a lot of things that I had suspected and explains a lot of things that I didn't understand (e.g why the texture zoom is ~0.71 in presets I've looked at)

The info that the textured option uses the output of the warp shader as the texture is a critical thing that I had missed, and explains a lot. Thanks!

No, I'm not interested in being compatible with milkdrop (I was surprised to learn that it's still a thing, I had assumed it died years ago with winamp), and I'd rather not mess around learning shader language. But you've confirmed my suspicion that this is how to achieve what I want to do. Thanks for the example preset, that's helpful. Maybe I'll give it a shot.

Most do it using GLSL

Are you saying that I can write shaders in GLSL? I wasn't sure about that. The docs say HLSL, but I suspected they were old and GLSL might be ok? The docs also say that projectm converts HLSL -> GLSL but that it's not 100%.

Aspect ratio correction in custom shapes is sadly broken beyond repair

Characterising the situation as "broken beyond repair" is quitter talk! ;) Surely it would be possible to have a variable in your preset file to tell projectm to correct the aspect ratio of textures, and then have this "do not touch" behaviour being conditional - it's on by default but if there's an ASPECT_CORRECT_TEXTURES=1 in the preset then it doesn't stretch the textures.

I just want to mention that it's only the textures which are being stretched, not the shapes themselves. If I don't texture the shape then it stays a square regardless of the screen size. I'm not sure whether this applies to just images loaded with ImageURL or to textures coming from the shader.

[AntiSol]

Hmmm, trying to run raycast.milk:

/home/antisol/code/raycast.milk(73) : Undeclared identifier 'distance'
Failed to parse HLSL(step2) Comp shader
Source: 
(shader source)

[OfficialIncubo]

Here is the fixed one, supported for majority of MilkDrop instances.

Here's how:

• break statement doesn't support on MilkDrop
• Fixed 'incorrect number of arguments to numeric type constructor' error. In HLSL, tries to read float3(1). but it fails. The thing it should work is float3(1,1,1). Same applies as float2, float4 and matrix constructors.
• distance() in HLSL works, here is the documentation: https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-distance

You can get this from here.

[AntiSol]

nope, sorry, same result. Here's the full output:

[PresetFactory] url is /home/antisol/code/raycast.milk
Successful compilation of Warp
/home/antisol/code/raycast.milk(73) : Undeclared identifier 'distance'
Failed to parse HLSL(step2) Comp shader
Source: 
uniform float4 texsize_pw_main;
uniform float4 texsize_pc_main;
uniform float4 texsize_noisevol_lq;
uniform float4 texsize_noisevol_hq;
uniform float4 texsize_noise_mq;
uniform float4 texsize_noise_lq_lite;
uniform float4 texsize_noise_lq;
uniform float4 texsize_noise_hq;
uniform float4 texsize_main;
uniform float4 texsize_fw_main;
uniform float4 texsize_fc_main;
uniform sampler2D sampler_pw_main;
uniform sampler2D sampler_pc_main;
uniform sampler3D sampler_noisevol_lq;
uniform sampler3D sampler_noisevol_hq;
uniform sampler2D sampler_noise_mq;
uniform sampler2D sampler_noise_lq_lite;
uniform sampler2D sampler_noise_lq;
uniform sampler2D sampler_noise_hq;
uniform sampler2D sampler_main;
uniform sampler2D sampler_fw_main;
uniform sampler2D sampler_fc_main;

uniform float4   rand_frame;		// random float4, updated each frame
uniform float4   rand_preset;   // random float4, updated once per *preset*
uniform float4   _c0;  // .xy: multiplier to use on UV's to paste an image fullscreen, *aspect-aware*; .zw = inverse.
uniform float4   _c1, _c2, _c3, _c4;
uniform float4   _c5;  //.xy = scale,bias for reading blur1; .zw = scale,bias for reading blur2; 
uniform float4   _c6;  //.xy = scale,bias for reading blur3; .zw = blur1_min,blur1_max
uniform float4   _c7;  // .xy ~= float2(1024,768); .zw ~= float2(1/1024.0, 1/768.0)
uniform float4   _c8;  // .xyzw ~= 0.5 + 0.5*cos(time * float4(~0.3, ~1.3, ~5, ~20))
uniform float4   _c9;  // .xyzw ~= same, but using sin()
uniform float4   _c10;  // .xyzw ~= 0.5 + 0.5*cos(time * float4(~0.005, ~0.008, ~0.013, ~0.022))
uniform float4   _c11;  // .xyzw ~= same, but using sin()
uniform float4   _c12;  // .xyz = mip info for main image (.x=#across, .y=#down, .z=avg); .w = unused
uniform float4   _c13;  //.xy = blur2_min,blur2_max; .zw = blur3_min, blur3_max.
uniform float4   _qa;  // q vars bank 1 [q1-q4]
uniform float4   _qb;  // q vars bank 2 [q5-q8]
uniform float4   _qc;  // q vars ...
uniform float4   _qd;  // q vars
uniform float4   _qe;  // q vars
uniform float4   _qf;  // q vars
uniform float4   _qg;  // q vars
uniform float4   _qh;  // q vars bank 8 [q29-q32]
// note: in general, don't use the current time w/the *dynamic* rotations!
uniform float4x3 rot_s1;  // four random, static rotations.  randomized @ preset load time.
uniform float4x3 rot_s2;  // minor translation component (<1).
uniform float4x3 rot_s3;
uniform float4x3 rot_s4;
uniform float4x3 rot_d1;  // four random, slowly changing rotations.
uniform float4x3 rot_d2;  
uniform float4x3 rot_d3;
uniform float4x3 rot_d4;
uniform float4x3 rot_f1;  // faster-changing.
uniform float4x3 rot_f2;
uniform float4x3 rot_f3;
uniform float4x3 rot_f4;
uniform float4x3 rot_vf1;  // very-fast-changing.
uniform float4x3 rot_vf2;
uniform float4x3 rot_vf3;
uniform float4x3 rot_vf4;
uniform float4x3 rot_uf1;  // ultra-fast-changing.
uniform float4x3 rot_uf2;
uniform float4x3 rot_uf3;
uniform float4x3 rot_uf4;
uniform float4x3 rot_rand1; // random every frame
uniform float4x3 rot_rand2;
uniform float4x3 rot_rand3;
uniform float4x3 rot_rand4;
float SphereSDF(float3 q)
{
    float r = 1.0;
    return distance(q, float3(0,0,0)) - r;
}

float CubeSDF(float3 q)
{
    float3 s = float3(.5,.5,.5);
    return length(max(abs(q) - s, 0));
}

float3 Projection(float2 p, float F)
{
    return normalize(float3(p.x, p.y, F));
}

float RayCast(float3 origin, float3 direction)
{
    float t = 0.0;

    for (int i = 0; i < 100; i++)
    {
        float3 q = origin + t * direction;
        float h = SphereSDF(q);

        if (h < 0.001)
        {
            
            h = 0; 
        }

        t += h;

        if (h > 20.0)
        {
            
            h = 0; 
        }
    }

    return t;
}

float3 CalcNormal(float3 q)
{
    float2 e = float2(0.0001, 0.0);
    return normalize(float3(
        SphereSDF(q + e.xyy) - SphereSDF(q - e.xyy),
        SphereSDF(q + e.yxy) - SphereSDF(q - e.yxy),
        SphereSDF(q + e.yyx) - SphereSDF(q - e.yyx)
    ));
}

void PS(float4 _vDiffuse : COLOR, float2 _uv : TEXCOORD0, float2 _rad_ang : TEXCOORD1, out float4 _return_value : COLOR)
 {
float3 ret = 0;

    float2 p =(_uv.xy) *(_c0);

    
    p = 2.0 * p - float2(1,1);

    
    float F = -1.5;
    float3 rayOrigin = float3(0.0, 0.0, 2.0);

    
    float3 rayDirection = Projection(p, F);

    
    
    float t = RayCast(rayOrigin, rayDirection);

    
    float3 color = float3(0,0,0);

    
    
    if (t < 20.0)
    {
        
        
        float3 q = rayOrigin + t * rayDirection;
        float3 normal = CalcNormal(q);
        color = normalize(normal);
    }

    ret = color;
_return_value = float4(ret.xyz, 1.0);
}
handleFailedPresetSwitch

It seems that just saying "distance is valid" wasn't enough to convince projectM of that ;)

To reiterate, I have no interest in making it work in milkdrop. I am only interested in projectM.

[OfficialIncubo]

Loading my fixed raycast preset to BeatDrop Music Visualizer had no issue at all, but I think the distance() and other undocumented functions from HLSL aren't implemented in projectM yet.... 🤔

Still working on writing undocumented functions...

[AntiSol]

It seems like this beatdrop is windows-only. Or in other words, totally useless to me because I haven't had a windows machine for over 20 years. I filed the issue against projectM and have specifically said that I'm only interested in projectM, so, with respect, I fail to see how "it works in a different program" qualifies as a response.

This is exactly the kind of issue that makes me not very keen to go writing shaders in HLSL.

[AntiSol]

I implemented the distance function in the shader code.

This compiles, fixing that message, but the display is just totally black.

code:

MILKDROP_PRESET_VERSION=201
PSVERSION=4
PSVERSION_WARP=4
PSVERSION_COMP=4
[preset00]
fRating=3.000
fGammaAdj=2.000
fDecay=0.980
fVideoEchoZoom=2.000
fVideoEchoAlpha=0.000
nVideoEchoOrientation=0
nWaveMode=0
bAdditiveWaves=0
bWaveDots=0
bWaveThick=0
bModWaveAlphaByVolume=0
bMaximizeWaveColor=1
bTexWrap=1
bDarkenCenter=0
bRedBlueStereo=0
bBrighten=0
bDarken=0
bSolarize=0
bInvert=0
fWaveAlpha=0.000
fWaveScale=1.000
fWaveSmoothing=0.750
fWaveParam=0.000
fModWaveAlphaStart=0.750
fModWaveAlphaEnd=0.950
fWarpAnimSpeed=0.000
fWarpScale=0.000
fZoomExponent=1.00000
fShader=0.000
zoom=1.00000
rot=0.00000
cx=0.500
cy=0.500
dx=0.00000
dy=0.00000
warp=0.00000
sx=1.00000
sy=1.00000
wave_r=1.000
wave_g=1.000
wave_b=1.000
wave_x=0.500
wave_y=0.500
ob_size=0.010
ob_r=0.000
ob_g=0.000
ob_b=0.000
ob_a=0.000
ib_size=0.010
ib_r=0.250
ib_g=0.250
ib_b=0.250
ib_a=0.000
nMotionVectorsX=12.000
nMotionVectorsY=9.000
mv_dx=0.000
mv_dy=0.000
mv_l=0.000
mv_r=1.000
mv_g=1.000
mv_b=1.000
mv_a=1.000
b1n=0.000
b2n=0.000
b3n=0.000
b1x=1.000
b2x=1.000
b3x=1.000
b1ed=0.250
warp_1=`shader_body
warp_2=`{
warp_3=`    // sample previous frame
warp_4=`    ret = tex2D( sampler_main, uv ).xyz;
warp_5=`    
warp_6=`    // darken (decay) over time
warp_7=`    ret *= 0.98; //or try: ret -= 0.004;
warp_8=`}
comp_1=`float distance(float3 a, float3 b)
comp_2=`{
comp_3=`return pow(b.x - a.x,2) + pow(b.y - a.y,2) + pow(b.z - a.z,2);
comp_4=`}
comp_5=`
comp_6=`float SphereSDF(float3 q)
comp_7=`{
comp_8=`float r = 1.0;
comp_9=`return distance(q, float3(0,0,0)) - r;
comp_10=`}
comp_11=`
comp_12=`float CubeSDF(float3 q)
comp_13=`{
comp_14=`float3 s = float3(.5,.5,.5);
comp_15=`return length(max(abs(q) - s, 0));
comp_16=`}
comp_17=`
comp_18=`float3 Projection(float2 p, float F)
comp_19=`{
comp_20=`return normalize(float3(p.x, p.y, F));
comp_21=`}
comp_22=`
comp_23=`float RayCast(float3 origin, float3 direction)
comp_24=`{
comp_25=`float t = 0.0;
comp_26=`
comp_27=`for (int i = 0; i < 100; i++)
comp_28=`{
comp_29=`float3 q = origin + t * direction;
comp_30=`float h = SphereSDF(q);
comp_31=`
comp_32=`if (h < 0.001)
comp_33=`{
comp_34=`//break; //Break statement doesn't work on MilkDrop.
comp_35=`h = 0; //So we replace like this.
comp_36=`}
comp_37=`
comp_38=`t += h;
comp_39=`
comp_40=`if (h > 20.0)
comp_41=`{
comp_42=`//break; //Break statement doesn't work on MilkDrop.
comp_43=`h = 0; //So we replace like this.
comp_44=`}
comp_45=`}
comp_46=`
comp_47=`return t;
comp_48=`}
comp_49=`
comp_50=`float3 CalcNormal(float3 q)
comp_51=`{
comp_52=`float2 e = float2(0.0001, 0.0);
comp_53=`return normalize(float3(
comp_54=`SphereSDF(q + e.xyy) - SphereSDF(q - e.xyy),
comp_55=`SphereSDF(q + e.yxy) - SphereSDF(q - e.yxy),
comp_56=`SphereSDF(q + e.yyx) - SphereSDF(q - e.yyx)
comp_57=`));
comp_58=`}
comp_59=`
comp_60=`shader_body {
comp_61=`float2 p = uv * aspect;
comp_62=`
comp_63=`// Center coordinate
comp_64=`p = 2.0 * p - float2(1,1);
comp_65=`
comp_66=`// Set focal length and camera position
comp_67=`float F = -1.5;
comp_68=`float3 rayOrigin = float3(0.0, 0.0, 2.0);
comp_69=`
comp_70=`// Find the ray direction
comp_71=`float3 rayDirection = Projection(p, F);
comp_72=`
comp_73=`// Calculate distance between
comp_74=`// the camera and the surface
comp_75=`float t = RayCast(rayOrigin, rayDirection);
comp_76=`
comp_77=`// Initialize color
comp_78=`float3 color = float3(0,0,0);
comp_79=`
comp_80=`// Set color to the normal if the object
comp_81=`// is not too far
comp_82=`if (t < 20.0)
comp_83=`{
comp_84=`// Find the point in the 3D space
comp_85=`// and the surface normal
comp_86=`float3 q = rayOrigin + t * rayDirection;
comp_87=`float3 normal = CalcNormal(q);
comp_88=`color = normalize(normal);
comp_89=`}
comp_90=`
comp_91=`ret = color;
comp_92=`}

this says:

[PresetFactory] url is /home/antisol/code/raycast.milk
Successful compilation of Warp
Successful compilation of Comp

But as I say the display is just totally black. AFAICT the distance function should be right. Not sure where to start looking or how to begin debugging.

[kblaschke]

It seems that just saying "distance is valid" wasn't enough to convince projectM of that ;)

No, the distance intrinsic was simply missing in the shader translator. It is supported in both HLSL and GLSL. It's also documented in MSDN: https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-distance

I had fixed it locally, but didn't push the change to master, sorry. It's now in there.

[kblaschke]

* break statement doesn't support on MilkDrop

Which is kinda interesting as the HLSL specs say it should be supported.

[OfficialIncubo]

...but why it doesn't work on MilkDrop?

[kblaschke]

...but why it doesn't work on MilkDrop?

Ah, this page here says it was introduced with Shader Model 4. DX9 only supports Shader Model 3 and below, SM4 was introduced in DX10. This is why.

Guess this means: don't use break in Milkdrop presets.

[OfficialIncubo]

The maximum pixel shader version is 4, that means the maximum shader model is 3, so that is. Porting BeatDrop/MilkDrop to DX10/DX11 using a shim is a bad plan?

[kblaschke]

Not sure what's up with the discord invite link but I just made a new one: https://discord.gg/N9DyQfCH4j

same deal: Whoops... Unable To Accept Invite

You can DM me on Discord (ID's codav), I may be able to invite you personally if that doesn't work.

[kblaschke]

Besides that, I guess this topic is rather two discussions (one of the OP and another regarding BeatDrop's DX support), so I'd suggest to either take it to Discord (if that works for everyone of course) or open up a discussion in our Q&A board, which is better suited for this than an actual issue.