DirectXShaderCompiler [SPIRV] Undefined function when compiling HLSL referencing external function to spirv library

I'm trying to compile spirv libraries from HLSL to be linked together at a later stage.

Exporting functions works as expected:

; SPIR-V
; Version: 1.0
; Generator: Google spiregg; 0
; Bound: 9
; Schema: 0
               OpCapability Shader
               OpCapability Linkage
               OpMemoryModel Logical GLSL450
               OpSource HLSL 610
               OpName %test1 "test1"
               OpName %bb_entry "bb.entry"
               OpName %test2 "test2"
               OpName %bb_entry_0 "bb.entry"
               OpDecorate %test1 LinkageAttributes "test1" Export
               OpDecorate %test2 LinkageAttributes "test2" Export
      %float = OpTypeFloat 32
%float_0_00999999978 = OpConstant %float 0.00999999978
%float_0_0199999996 = OpConstant %float 0.0199999996
          %8 = OpTypeFunction %float
      %test1 = OpFunction %float None %8
   %bb_entry = OpLabel
               OpReturnValue %float_0_00999999978
               OpFunctionEnd
      %test2 = OpFunction %float None %8
 %bb_entry_0 = OpLabel
               OpReturnValue %float_0_0199999996
               OpFunctionEnd

When I try to build the main shader with an entry point, the compiler complains about a function not being defined:

float4 LibFunc();
            
            [shader("pixel")]
            float4 PSMain() : SV_Target0
            {
                return LibFunc();
            }

test.hlsl:6:12: error: found undefined function return LibFunc();

This is the command line I'm using: dxc test.hlsl -Ges -Fo output.spv -Fc output.txt -T lib_6_1 -Vd -spirv

When I try to compile to DXIL, it seems to work as expected: dxc test.hlsl -Ges -Fo output.spv -Fc output.txt -T lib_6_1 -Vd

[...]
declare <4 x float> @"\01?LibFunc@@YA?AV?$vector@M$03@@XZ"()

define void @PSMain() {
  %1 = call <4 x float> @"\01?LibFunc@@YA?AV?$vector@M$03@@XZ"() #0
  [...]

From looking at the code that emits spirv, I can see that it's not taking into account the case in which the definition of the function is not provided:

  const FunctionDecl *callee = getCalleeDefinition(callExpr);

  // Note that we always want the defintion because Stmts/Exprs in the
  // function body references the parameters in the definition.
  if (!callee) {
    emitError("found undefined function", callExpr->getExprLoc());
    return nullptr;
  }

It seems to me the logic could look something like (pseudo code):

if (!callee && is_not_a_lib()) {
  // error
} else {
  // we are in a lib, emit a call instruction to the function decorated with import linkage attribute
  // OpDecorate %test1 LinkageAttributes "<func_name>" Import
}

https://www.khronos.org/registry/SPIR-V/specs/1.0/SPIRV.html#Linkage_Type

This is the call that can be used to decorate a target function declaration:

  /// \brief Decorates the given target with LinkageAttributes
  /// We have to set targetInst as nullptr when it is an imported or exported
  /// function.
  /// We have to set targetFunc as nullptr when it is an imported or
  /// exported global variable.
  void decorateLinkage(SpirvInstruction *targetInst, SpirvFunction *targetFunc,
                       llvm::StringRef name, spv::LinkageType linkageType,
                       SourceLocation);

Which is already being used for exported functions:

  SpirvFunction *spirvFunction = spvBuilder.createSpirvFunction(
      fn->getReturnType(), fn->getLocation(),
      getFunctionOrOperatorName(fn, true), isPrecise, isNoInline);

  if (fn->getAttr<HLSLExportAttr>()) {
    spvBuilder.decorateLinkage(nullptr, spirvFunction, fn->getName(),
                               spv::LinkageType::Export, fn->getLocation());
  }

Apr 03 '22 22:04 fcarucci

Hi @jaebaek! One for you

Apr 20 '22 18:04 pow2clk

Hey Jaebaeck, I think we’ve met. I was in Android Graphics working with Nat Duca until very recently.

On Apr 20, 2022, at 11:36 AM, Greg Roth @.***> wrote:

Hi @jaebaek https://github.com/jaebaek! One for you

— Reply to this email directly, view it on GitHub https://github.com/microsoft/DirectXShaderCompiler/issues/4373#issuecomment-1104322991, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYI6WXAVCPXPJZAQAMYV3G3VGBFENANCNFSM5SNZ23AQ. You are receiving this because you authored the thread.

Apr 21 '22 01:04 fcarucci

@fcarucci Thank you for reporting this issue. So far we have not tested compiling shaders with the external linkage. Your solution looks good to me.

If you made contributions for that part, we would appreciate it. We currently experience huge amount of workload and I am not sure we can handle it for a while. I am sorry about that.

Note that we need the legalization to generate a valid SPIR-V and external linkage can generate invalid SPIR-V because we cannot run spirv-opt legalization on shaders with the external linkage. Function inlining is one of the most important parts of the legalization. Without specifying the external function in the same SPIR-V module, we cannot run the function inlining pass.

So .. we you can do will be

Support the external linkage in DXC
generate SPIR-V using DXC with -Vd -fcgl that means "turn off validation (spirv-val), turn off optimization/legalization"
link your shader module (using spirv-link) and make sure your shader doesn't have missing functions
run spirv-opt to legalize (or optimize) it.

Apr 21 '22 20:04 jaebaek

Thanks for getting back to me.

If I understand it correctly, 2) should let me compile correctly and then I can proceed with linking?

On Apr 21, 2022, at 1:14 PM, Jaebaek Seo @.***> wrote:

@fcarucci https://github.com/fcarucci Thank you for reporting this issue. So far we have not tested compiling shaders with the external linkage. Your solution looks good to me.

If you made contributions for that part, we would appreciate it. We currently experience huge amount of workload and I am not sure we can handle it for a while. I am sorry about that.

Note that we need the legalization to generate a valid SPIR-V and external linkage can generate invalid SPIR-V because we cannot run spirv-opt legalization on shaders with the external linkage. Function inlining is one of the most important parts of the legalization. Without specifying the external function in the same SPIR-V module, we cannot run the function inlining pass.

So .. we you can do will be

Support the external linkage in DXC generate SPIR-V using DXC with -Vd -fcgl that means "turn off validation (spirv-val), turn off optimization/legalization" link your shader module (using spirv-link) and make sure your shader doesn't have missing functions run spirv-opt to legalize (or optimize) it. — Reply to this email directly, view it on GitHub https://github.com/microsoft/DirectXShaderCompiler/issues/4373#issuecomment-1105713211, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYI6WXHFMWI67O23NZUV7MTVGGZJTANCNFSM5SNZ23AQ. You are receiving this because you were mentioned.

Oct 11 '22 07:10 fcarucci