dxil-redist: new package

[lb90]

Contains copies of the dxil.dll redistributable libraries from the Windows SDK.

As dxil.dll is not present in system installations, we have to ship it ourselves.

Note that another official distribution channel for dxil.dll is https://github.com/microsoft/DirectXShaderCompiler/releases, but we avoid using that because the distributed modules are dependant upon the Visual C++ runtime, while the modules from the Windows SDK are built with the Visual C++ runtime linked statically.

~~TODO:~~ figure out a valid SPDX license id? ✔️

See also: https://wiki.archlinux.org/title/Nonfree_applications_package_guidelines

[lazka]

sooo... what's the license and can we redistribute it.. and how do you know your binaries are from the same source?

[lb90]

Hi @lazka! The redistributables are not under any of the standard licenses, but a custom one outlined in %WINSDK%\Licenses\%VER%\sdk_license.rtf. The latest WinSDK as of today is 10.0.22621.0 and here's the relevant excerpt from sdk_license.rtf:

    3. ADDITIONAL LICENSING REQUIREMENTS AND/OR USE RIGHTS.
        a. Distributable Code. The software contains code that you are permitted to distribute in programs you develop if you comply with the terms below.
            i. Right to Use and Distribute. The code and test files listed below are “Distributable Code”.
                • REDIST.TXT Files. You may copy and distribute the object code form of code listed in REDIST.TXT files plus the files listed on the REDIST.TXT list located at http://go.microsoft.com/fwlink/?LinkId=524842. Depending on the specific edition of the software, the number of REDIST files you receive with the software may not be equal to the number of REDIST files listed in the REDIST.TXT List. We may add additional files to the list from time to time.
                • Third Party Distribution. You may permit distributors of your programs to copy and distribute the Distributable Code as part of those programs. 
            ii. Distribution Requirements. For any Distributable Code you distribute, you must
                • Add significant primary functionality to it in your programs;
                • For any Distributable Code having a filename extension of .lib, distribute only the results of running such Distributable Code through a linker with your program;
                • Distribute Distributable Code included in a setup program only as part of that setup program without modification;
                • Require distributors and external end users to agree to terms that protect it at least as much as this agreement;
                • For Distributable Code from the Windows Performance Toolkit portions of the software, distribute the unmodified software package as a whole with your programs, with the exception of the KernelTraceControl.dll and the WindowsPerformanceRecorderControl.dll which can be distributed with your programs;
                • Display your valid copyright notice on your programs; and
                • Indemnify, defend, and hold harmless Microsoft from any claims, including attorneys’ fees, related to the distribution or use of your programs. 
            iii. Distribution Restrictions. You may not
                • Alter any copyright, trademark or patent notice in the Distributable Code;
                • Use Microsoft’s trademarks in your programs’ names or in a way that suggests your programs come from or are endorsed by Microsoft;
                • Distribute partial copies of the Windows Performance Toolkit portion of the software package with the exception of the KernelTraceControl.dll and the WindowsPerformanceRecorderControl.dll which can be distributed with your programs;
                • Distribute Distributable Code to run on a platform other than the Microsoft operating system platform;
                • Include Distributable Code in malicious, deceptive or unlawful programs; or
                • Modified or distribute the source code of any Distributable Code so that any part of it becomes subject to an Excluded License. And Excluded License is one that requires, as a condition of use, modification or distribution, that
                        ▪ The code be disclosed or distributed in source code form; or
                        ▪ Others have the right to modify it.

Yes, dxil.dll is redistributable as mentioned in http://go.microsoft.com/fwlink/?LinkId=524842

---

The binaries were taken from my local copy of the Windows SDK: C:\Program Files (x86)\Windows Kits\10\Redist\D3D\{arm64,x64,x86}\dxil.dll (the latest version, of course: 10.0.22621.0); however the files are digitally signed by Microsoft, so we can rest assured that the files have not been tampered with. BTW, if feasible I'd add a check() for the digital signature

The actual version is reported in the PE version information resource: 101.6.2112.13 (release/github-release-1.6.2112, b9cfa8a) for all the three DLLs.

Actually I'd change the package version to match that instead of the WinSDK version. That's better because in the future we might switch to https://github.com/microsoft/DirectXShaderCompiler/releases as the distribution channel.

[driver1998]

We should provide hashes to the DLLs so that people can at least verify.

Also are there other release channels instead of copying from Windows SDK? Nuget maybe?

[lb90]

Searched for nuget packages, but couldn't find anything. I also don't like including binaries in this git repo. Can we somehow host the binaries on our own? Maybe on GitHub using GitHub Pages? Or even create our own nuget package?

[lazka]

Distribute Distributable Code included in a setup program only as part of that setup program without modification;

Is this the case here?

Require distributors and external end users to agree to terms that protect it at least as much as this agreement;

This means we need to make users agree to the terms I guess? I don't see how we can do that via pacman. Unless an install scripts that prints the requirements is enough. Or a script that downloads the files after the user agrees.

Either way, I don't feel comfortable committing DLLs in this repo (for performance and legal reasons), so this needs to be in some external repo which we reference.

[lb90]

Is this the case here?

Yes, strip is avoided and we check for the Microsoft signature, so no modification is done to the DLL.

This means we need to make users agree to the terms I guess?

Yeah. As a matter of fact almost nobody shows the EULA agreement; most of the times vcredist is launched with the /q (quiet) argument, or the redistributable DLLs are included in an archive which is extracted manually by the end user. And even on https://github.com/microsoft/DirectXShaderCompiler/releases there's no sign of an EULA 🙂 Anyway that's not an excuse for us to do the same, and we should look forward into that.

[driver1998]

I opened an issue to request statically-linked releases: https://github.com/microsoft/DirectXShaderCompiler/issues/5163

[lb90]

Hi @dvrogozh! We should figure out a way to make users accept an EULA, probably by modifying pacman

[dvrogozh]

Well, ok, I agree EULA is a bigger issue which needs to be fixed first. In a meanwhile, current problem with missing dxil.dll is that some workflows will simply crash if dxil.dll is missing. However, not all such workflows do need dxil.dll. Thus I suggest that resolving such a crash is essential. I prepared a PR #17421 with @lb90 patch in mesa main - hope it can be merged.

[lazka]

@lb90 are you happy with #17421 or is this still needed?

[driver1998]

Well is dxil.dll not needed for GLon12? Otherwise blender will crash if you have mesa installed (which installs itself as $MINGW_PREFIX/bin/opengl32.dll).

[dvrogozh]

In a long term I think that's needed. For the VAon12 part, having no no dxil.dll just limits a number of supported use cases. Which is only hardware supported media operations will be functional and those operations which are not supported in media hardware will require shaders and, hence, dxil.dll. In such a case hope is that driver will gracefully exit with appropriate errors instead of crash though I am really not sure that was thoroughly tested.

For GLon12 I don't know to which degree it requires dxil.dll. Potentially it might be that GLon12 won't make sense without it, but we need and expert to answer on that.

[driver1998]

Not that you should install mesa on MSYS2 as-is unless absolutely necessary though. Since it will make all the exes in $MINGW_PREFIX/bin use mesa OpenGL, which usually is GLon12.

[raedrizqie]

i think we can merge this PR.. dxc.exe requires dxil.dll at runtime, to run smoothly.

i am currently fixing dx shader compiler to get compiled on mingw.. only one thing left.. implementation of __uuidof() is not compatible with MSVC's

[lb90]

For prompting an EULA we may use a pre_install script. But we should also retain unattended installations via pacman --noconfirm. Should --noconfirm imply agreements of EULA or not?

Actually a few projects targeting Linux are subject to EULA. An example is the Vivante graphics driver. However the Arch Linux ARM package doesn't prompt for an EULA: https://archlinuxarm.org/packages/armv7h/imx-gpu-viv-g2d/files/PKGBUILD

[lb90]

Other interesting examples:

1. minecraft (see section 4.2.3)
2. hsa-amd-aqlprofile-bin
3. https://bbs.archlinux.org/viewtopic.php?id=58622
4. matlab

[driver1998]

https://github.com/microsoft/DirectXShaderCompiler/issues/5163#event-9968943376:

Fixed in internal build system. Next DXC release will ship with hybrid CRT linkage.

Hybrid CRT linkage means it statically links to Visual C++ runtime, but dynamically links to the UCRT. Since dxil.dll is only useful in Windows 10+, this should be fine. If mesa loads dxil.dll on Windows 8.x and lower, we might need to add a OS version check before this.

[driver1998]

https://github.com/microsoft/DirectXShaderCompiler/releases/tag/v1.7.2308

v1.7.2308 (August 2023) just released, and here is the imports of dxil.dll downloaded from GitHub releases:

driver1998@desktop UCRT64 /d/Documents/Downloads/dxc_2023_08_14/bin/x64
$ ldd dxil.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffac0190000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffabf3f0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffabd8a0000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffabfe70000)
        ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x7ffabe130000)
        sechost.dll => /c/WINDOWS/System32/sechost.dll (0x7ffabdf50000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffabea80000)
        ole32.dll => /c/WINDOWS/System32/ole32.dll (0x7ffabe410000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ffabd800000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffabdc50000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ffac0120000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ffabdd70000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ffabd570000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ffabe6e0000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x7ffabf060000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x7ffabf020000)

[hmartinez82]

Someone was asking about NuGet packages. I don't know how long ago, but MS started deploying the Windows SDK as a NuGet package. So now it's possible to get that file, for all archs, from https://www.nuget.org/packages/Microsoft.Windows.SDK.CPP/

It's just a zip file. The location within is \c\Redist\D3D\{arch}\dxil.dll

[lazka]

A somewhat related blog post: https://devlog.hexops.com/2024/building-the-directx-shader-compiler-better-than-microsoft/