reshade-steam-proton icon indicating copy to clipboard operation
reshade-steam-proton copied to clipboard

Install Reshade for a Vulkan game

Open rylydou opened this issue 3 years ago • 63 comments

I am installing Reshade into Cemu which uses Vulkan. I know that on Windows you can install Reshade on a Vulkan game. But I don't know what dll name to use or if it is even possible.

I have tried:

  • dxgi - The suggested option
  • opengl32 - Because Cemu has an option to switch to that but I would prefer to use Vulkan for performance
  • vulkan

rylydou avatar Jan 16 '22 16:01 rylydou

I'm not sure, never used Cemu.

While doing a quick search I found this: https://i.imgur.com/o8nwDkd.jpg linked here https://teddit.net/r/CemuPiracy/comments/ey6ien/is_there_any_tutorial_for_reshade_on_cemu_botw/

Does that mean the games need patches for Reshade to function?

Looking at this video at 4m50s, there's no dll file in this Cemu folder, I'm guessing Reshade does something different to hook into games using Vulkan?

kevinlekiller avatar Jan 16 '22 16:01 kevinlekiller

ReShade does not support DLL wrapping in Vulkan. Vulkan has a layering mechanism that ReShade has to use to be spec conform. And that requires ReShade to be in a central place, because those layers are configured via system-wide registry keys. You are best off using the setup tool to set up ReShade, it knows what to do to make it work.

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan

I'm guessing if we figure out which registrey keys are added, then we could try to do that programatically?

kevinlekiller avatar Jan 16 '22 17:01 kevinlekiller

For now, you could use vkBasalt which supports most Reshade shaders, only issue is there's no in game GUI.

kevinlekiller avatar Jan 16 '22 17:01 kevinlekiller

This post explains how the installer sets up the registry:

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan#41351

kevinlekiller avatar Jan 16 '22 17:01 kevinlekiller

Thanks, I'll give vkBasalt with goverlay a try. Thanks for the help!

rylydou avatar Jan 16 '22 17:01 rylydou

Not tested in a game, but I've got something that might work if you want to test.

The WINEPREFIX has to be set for the game in particular and the ReShade64.json needs to be changded to ReShade32.json if the program is 32 bits, the reg:64 also has to be changed to reg:32 if it's a 32 bit prefix.

WINEPREFIX="$HOME/Games/windows/lutris/Test/wineprefix" wine reg ADD HKLM\\SOFTWARE\\Khronos\\Vulkan\\ImplicitLayers /d 0 /t REG_DWORD /v "Z:\\home\\$USER\\.reshade\\reshade\\ReShade64.json" -f /reg:64

Here's regedit in a test WINEPREFIX: https://i.imgur.com/80KiVki.png

kevinlekiller avatar Jan 16 '22 23:01 kevinlekiller

I've have tried doing the registry edit but I don't have either of the ReShade dlls in ~/.reshade/reshade. I do have the json files that point to them though.

rylydou avatar Jan 17 '22 00:01 rylydou

That's a good point, the dxgi.dll would have to be linked to ReShade64.dll and the d3d9.dll to ReShade32.dll

I'll add those things to the script, will have to find a Vulkan game to test on.

kevinlekiller avatar Jan 17 '22 01:01 kevinlekiller

Half life 2 is 32 bit and has Vulkan support : https://www.pcgamingwiki.com/wiki/Half-Life_2#API

Doom 2016 64 bit / Vulkan : https://www.pcgamingwiki.com/wiki/Doom_(2016)#API

Will test these this week and update the script.

kevinlekiller avatar Jan 17 '22 01:01 kevinlekiller

I've been testing on Doom, but not having any success, the registry entry is correctly added, but the game doesn't launch with Reshade.

Added the changes to a different branch:

https://github.com/kevinlekiller/reshade-steam-proton/commit/8275417a3e0d4c18ee1042d5c0c8675b33b13fac

kevinlekiller avatar Jan 18 '22 20:01 kevinlekiller

In a Windows VM I installed Reshade to make sure the Registry matches what's in wine:

Screenshot_20220118_155049

So the issue is somewhere else.

kevinlekiller avatar Jan 18 '22 20:01 kevinlekiller

The issue might be with Reshade itself:

https://reshade.me/forum/suggestions/7074-linux-vulkan-opengl-version#40598

This unlikely to happen anytime soon, since large parts of the ReShade code base are Windows specific (especially the hooking code). Only thing that could potentially be separated out would be the Vulkan layer implementation, but that is already partially covered by vkBasalt so it doesn't feel worth the effort right now.

Edit: Although, (in our case) it's through Wine, but perhaps there's some Windows specific Vulkan stuff that's not available on Linux so Wine can't do anything about it?

kevinlekiller avatar Jan 18 '22 20:01 kevinlekiller

In ReShade 5.0, there are some changes to how the json files are handled.

Here's the relevant section:

https://github.com/crosire/reshade/blob/67f4f718614800b426f20291b2fc7d7ca6644f37/setup/MainWindow.xaml.cs#L711

kevinlekiller avatar Feb 01 '22 19:02 kevinlekiller

Would it be possible to install Reshade 4.9.1 with global Vulkan enabling option? This is better than nothing for now.

canceralp avatar Jun 07 '22 16:06 canceralp

Not sure, from what I can remember it doesn't function at all. Busy with work and other stuff at the moment, so not enough time to test, unfortunately.

kevinlekiller avatar Jun 10 '22 21:06 kevinlekiller

I tried 4.9.1, 5.0.2 and 5.2.1. None works. I tried them on Valheim and RDR 2. Valheim can also work on Dx11 so it is possible to make the installation for it but RDR 2 has only Vulkan for Linux (Dx12 freezes the PC).

I have read somewhere Wine runs all the games in administrator mode. I also know that Reshade Vulkan doesn't load when the game is run with admin privileges. Could that be the problem?

canceralp avatar Jun 11 '22 09:06 canceralp

It is not about Vulkan games but there is good news from Crosire. He fixed a problem which prevented newer Reshade versions to work with DirectX 12 games. I hope he might look into Vulkan problem as well.

canceralp avatar Aug 22 '22 22:08 canceralp

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

SirSAC avatar Sep 06 '23 18:09 SirSAC

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

canceralp avatar Sep 08 '23 06:09 canceralp

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

I tried this on Baldur's Gate 3 on my Linux machine where I do have Reshade working, but it did not work.

screwylightbulb avatar Sep 13 '23 12:09 screwylightbulb

I tried too for Borderland 2 (DX9) and TerraTech (DX11) and won't work, so sorry, it was just an idea, but works only if I run the setup and select the game executable and telling the setup is running on Vulkan, is creating "ReShade32.dll" or "ReShade64.dll" but not the JSON file in the game folder and something else is creating in "C:\ProgramData" that contain the original JSON file and maybe something in registry but I'm not able yet to track what is it or if is something added/changed. My test PC with Linux died :( sorry if I'm not able to provide more details, I just give what I remember from the Linux machine.

PS: try to run the setup of ReShade and give the game executable and select Vulkan for runtime, maybe this will work if setup takes the details correctely.

SirSAC avatar Sep 20 '23 06:09 SirSAC

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

EndlesslyFlowering avatar Dec 29 '23 12:12 EndlesslyFlowering

This is early new year:) I am (and many more, I'm sure) looking forward to instructions.

canceralp avatar Dec 29 '23 18:12 canceralp

it works both normally and through gamescope. really tired now. will post how to do this later :)

I will gladly use those instructions! After you're well rested of course :) As I managed thanks to your preview DLL and some help from the community to have an HDR signal in System Shock 2! This might be the path towards universal HDR support on SteamOS!

Deus-nsf avatar Dec 29 '23 22:12 Deus-nsf

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

EndlesslyFlowering avatar Dec 31 '23 04:12 EndlesslyFlowering

this may take a long while...

I'm not in a hurry, it was great to have a proof of concept so quick already :) Better take the time and do things properly :)

Deus-nsf avatar Dec 31 '23 12:12 Deus-nsf

took a while to figure out: doom_reshade it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

follow-up?

beckkake avatar Feb 24 '24 19:02 beckkake