blade icon indicating copy to clipboard operation
blade copied to clipboard

Crash at start on Snapdragon X elite

Open HgGamer opened this issue 1 year ago • 18 comments

Vulkan sdk is installed, vkcube is working fine.

C:\Development\blade>cargo run --example bunnymark
warning: `C:\Development\blade\.cargo\config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
warning: use of deprecated method `winit::event_loop::EventLoop::<T>::create_window`: use `ActiveEventLoop::create_window` instead
   --> examples\bunnymark\main.rs:371:29
    |
371 |     let window = event_loop.create_window(window_attributes).unwrap();
    |                             ^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

warning: use of deprecated method `winit::event_loop::EventLoop::<T>::run`: use `EventLoop::run_app` instead
   --> examples\bunnymark\main.rs:400:10
    |
400 |         .run(|event, target| {
    |          ^^^

warning: `blade` (example "bunnymark") generated 2 warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
     Running `target\debug\examples\bunnymark.exe`
DeviceInformation { is_software_emulated: false, device_name: "Qualcomm(R) Adreno(TM) X1-85 GPU", driver_name: "Qualcomm Technologies Inc. Adreno Vulkan Driver", driver_info: "Driver Build: 36a15f63c3, , 1727332442\nDate: 09/26/2024\nCompiler Version: E031.47.15.00\nDriver Branch: \n" }
thread 'main' panicked at blade-graphics\src\vulkan\pipeline.rs:601:18:
called `Result::unwrap()` on an `Err` value: ([0x0], ERROR_UNKNOWN)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\examples\bunnymark.exe` (exit code: 101)

HgGamer avatar Dec 11 '24 07:12 HgGamer

Interesting, thank you for the report! Could you run with validation enabled? I.e. have the validation layers installed and do "validation: true" in the context desc.

kvark avatar Dec 13 '24 06:12 kvark

I changed the validation to true then false in bunnymark/main.rs:82

I still get the same output. Note, I'm not a rust developer

HgGamer avatar Dec 13 '24 08:12 HgGamer

It looks like it fails at vkCreateGraphicsPipelines(). Most examples (except init, mini) fails at this specific call.

Thread 0, Frame 0:
vkCreateGraphicsPipelines(device, pipelineCache, createInfoCount, pCreateInfos, pAllocator, pPipelines) returns VkResult VK_ERROR_UNKNOWN (-13):
    device:                         VkDevice = 000001EFE0839220
    pipelineCache:                  VkPipelineCache = 0000000000000000
    createInfoCount:                uint32_t = 1
    pCreateInfos:                   const VkGraphicsPipelineCreateInfo* = 0000009416741790
        pCreateInfos[0]:                const VkGraphicsPipelineCreateInfo = 0000009416741790:
            sType:                          VkStructureType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO (28)
            pNext:                          const void* = VkPipelineRenderingCreateInfo
            flags:                          VkPipelineCreateFlags = 0
            stageCount:                     uint32_t = 2
            pStages:                        const VkPipelineShaderStageCreateInfo* = 00000094167401E0
                pStages[0]:                     const VkPipelineShaderStageCreateInfo = 00000094167401E0:
                    sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO (18)
                    pNext:                          const void* = NULL
                    flags:                          VkPipelineShaderStageCreateFlags = 0
                    stage:                          VkShaderStageFlagBits = 1 (VK_SHADER_STAGE_VERTEX_BIT)
                    module:                         VkShaderModule = E7E6D0000000000F
                    pName:                          const char* = "vs_main"
                    pSpecializationInfo:            const VkSpecializationInfo* = NULL
                pStages[1]:                     const VkPipelineShaderStageCreateInfo = 0000009416740210:
                    sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO (18)
                    pNext:                          const void* = NULL
                    flags:                          VkPipelineShaderStageCreateFlags = 0
                    stage:                          VkShaderStageFlagBits = 16 (VK_SHADER_STAGE_FRAGMENT_BIT)
                    module:                         VkShaderModule = D5B26F0000000010
                    pName:                          const char* = "fs_main"
                    pSpecializationInfo:            const VkSpecializationInfo* = NULL
            pVertexInputState:              const VkPipelineVertexInputStateCreateInfo* = 00000094167403C0:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO (19)
                pNext:                          const void* = NULL
                flags:                          VkPipelineVertexInputStateCreateFlags = 0
                vertexBindingDescriptionCount:  uint32_t = 0
                pVertexBindingDescriptions:     const VkVertexInputBindingDescription* = 0000000000000004
                vertexAttributeDescriptionCount: uint32_t = 0
                pVertexAttributeDescriptions:   const VkVertexInputAttributeDescription* = 0000000000000004
            pInputAssemblyState:            const VkPipelineInputAssemblyStateCreateInfo* = 0000009416740450:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO (20)
                pNext:                          const void* = NULL
                flags:                          VkPipelineInputAssemblyStateCreateFlags = 0
                topology:                       VkPrimitiveTopology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST (3)
                primitiveRestartEnable:         VkBool32 = 0
            pTessellationState:             const VkPipelineTessellationStateCreateInfo* = NULL
            pViewportState:                 const VkPipelineViewportStateCreateInfo* = 00000094167406C0:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO (22)
                pNext:                          const void* = NULL
                flags:                          VkPipelineViewportStateCreateFlags = 0
                viewportCount:                  uint32_t = 1
                pViewports:                     const VkViewport* = UNUSED
                scissorCount:                   uint32_t = 1
                pScissors:                      const VkRect2D* = UNUSED
            pRasterizationState:            const VkPipelineRasterizationStateCreateInfo* = 00000094167404B0:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO (23)
                pNext:                          const void* = NULL
                flags:                          VkPipelineRasterizationStateCreateFlags = 0
                depthClampEnable:               VkBool32 = 0
                rasterizerDiscardEnable:        VkBool32 = 0
                polygonMode:                    VkPolygonMode = VK_POLYGON_MODE_FILL (0)
                cullMode:                       VkCullModeFlags = 0 (VK_CULL_MODE_NONE)
                frontFace:                      VkFrontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE (0)
                depthBiasEnable:                VkBool32 = 0
                depthBiasConstantFactor:        float = 0
                depthBiasClamp:                 float = 0
                depthBiasSlopeFactor:           float = 0
                lineWidth:                      float = 1
            pMultisampleState:              const VkPipelineMultisampleStateCreateInfo* = 0000009416740788:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO (24)
                pNext:                          const void* = NULL
                flags:                          VkPipelineMultisampleStateCreateFlags = 0
                rasterizationSamples:           VkSampleCountFlagBits = 1 (VK_SAMPLE_COUNT_1_BIT)
                sampleShadingEnable:            VkBool32 = 0
                minSampleShading:               float = 0
                pSampleMask:                    const VkSampleMask* = 0000009416740780
                    pSampleMask[0]:                 const VkSampleMask = 4294967295
                alphaToCoverageEnable:          VkBool32 = 0
                alphaToOneEnable:               VkBool32 = 0
            pDepthStencilState:             const VkPipelineDepthStencilStateCreateInfo* = 0000009416740850:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO (25)
                pNext:                          const void* = NULL
                flags:                          VkPipelineDepthStencilStateCreateFlags = 0
                depthTestEnable:                VkBool32 = 0
                depthWriteEnable:               VkBool32 = 0
                depthCompareOp:                 VkCompareOp = VK_COMPARE_OP_NEVER (0)
                depthBoundsTestEnable:          VkBool32 = 0
                stencilTestEnable:              VkBool32 = 0
                front:                          VkStencilOpState = 0000009416740878:
                    failOp:                         VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    passOp:                         VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    depthFailOp:                    VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    compareOp:                      VkCompareOp = VK_COMPARE_OP_NEVER (0)
                    compareMask:                    uint32_t = 0
                    writeMask:                      uint32_t = 0
                    reference:                      uint32_t = 0
                back:                           VkStencilOpState = 0000009416740894:
                    failOp:                         VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    passOp:                         VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    depthFailOp:                    VkStencilOp = VK_STENCIL_OP_KEEP (0)
                    compareOp:                      VkCompareOp = VK_COMPARE_OP_NEVER (0)
                    compareMask:                    uint32_t = 0
                    writeMask:                      uint32_t = 0
                    reference:                      uint32_t = 0
                minDepthBounds:                 float = 0
                maxDepthBounds:                 float = 0
            pColorBlendState:               const VkPipelineColorBlendStateCreateInfo* = 0000009416740F80:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO (26)
                pNext:                          const void* = NULL
                flags:                          VkPipelineColorBlendStateCreateFlags = 0
                logicOpEnable:                  VkBool32 = 0
                logicOp:                        VkLogicOp = VK_LOGIC_OP_CLEAR (0)
                attachmentCount:                uint32_t = 1
                pAttachments:                   const VkPipelineColorBlendAttachmentState* = 000001EFE0E89EE0
                    pAttachments[0]:                const VkPipelineColorBlendAttachmentState = 000001EFE0E89EE0:
                        blendEnable:                    VkBool32 = 1
                        srcColorBlendFactor:            VkBlendFactor = VK_BLEND_FACTOR_ONE (1)
                        dstColorBlendFactor:            VkBlendFactor = VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA (7)
                        colorBlendOp:                   VkBlendOp = VK_BLEND_OP_ADD (0)
                        srcAlphaBlendFactor:            VkBlendFactor = VK_BLEND_FACTOR_ONE_MINUS_DST_ALPHA (9)
                        dstAlphaBlendFactor:            VkBlendFactor = VK_BLEND_FACTOR_ONE (1)
                        alphaBlendOp:                   VkBlendOp = VK_BLEND_OP_ADD (0)
                        colorWriteMask:                 VkColorComponentFlags = 15 (VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT)
                blendConstants:                 float[4] = 0000009416740FA8
                    blendConstants[0]:              float = 0
                    blendConstants[1]:              float = 0
                    blendConstants[2]:              float = 0
                    blendConstants[3]:              float = 0
            pDynamicState:                  const VkPipelineDynamicStateCreateInfo* = 0000009416740680:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO (27)
                pNext:                          const void* = NULL
                flags:                          VkPipelineDynamicStateCreateFlags = 0
                dynamicStateCount:              uint32_t = 4
                pDynamicStates:                 const VkDynamicState* = 0000009416740670
                    pDynamicStates[0]:              const VkDynamicState = VK_DYNAMIC_STATE_VIEWPORT (0)
                    pDynamicStates[1]:              const VkDynamicState = VK_DYNAMIC_STATE_SCISSOR (1)
                    pDynamicStates[2]:              const VkDynamicState = VK_DYNAMIC_STATE_BLEND_CONSTANTS (4)
                    pDynamicStates[3]:              const VkDynamicState = VK_DYNAMIC_STATE_STENCIL_REFERENCE (8)
            layout:                         VkPipelineLayout = DD3A8A0000000015
            renderPass:                     VkRenderPass = 0000000000000000
            subpass:                        uint32_t = 0
            basePipelineHandle:             VkPipeline = 0000000000000000
            basePipelineIndex:              int32_t = 0
        pNext:                          VkPipelineRenderingCreateInfo = 0000009416740FF0:
            sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_RENDERING_CREATE_INFO (1000044002)
            pNext:                          const void* = NULL
            viewMask:                       uint32_t = 0
            colorAttachmentCount:           uint32_t = 1
            pColorAttachmentFormats:        const VkFormat* = 000001EFE01F3AB0
                pColorAttachmentFormats[0]:     const VkFormat = VK_FORMAT_B8G8R8A8_SRGB (50)
            depthAttachmentFormat:          VkFormat = VK_FORMAT_D32_SFLOAT_S8_UINT (130)
            stencilAttachmentFormat:        VkFormat = VK_FORMAT_D32_SFLOAT_S8_UINT (130)
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pPipelines:                     VkPipeline* = 000001EFE01F3B30
        pPipelines[0]:                  VkPipeline = 0000000000000000

There's no any errors spotted by the validation layer. I guess it's something with quirks in Qualcomm's Vulkan driver on Windows. Didn't have this problem on Linux (mesa/turnip).

segfault87 avatar Dec 18 '24 01:12 segfault87

It's really hard to say what part of the shader the Qualcomm's Vulkan driver doesn't like, but it looks like a driver bug to me.

kvark avatar Dec 19 '24 05:12 kvark

It's really hard to say what part of the shader the Qualcomm's Vulkan driver doesn't like, but it looks like a driver bug to me.

I'm an experienced Rust dev w/ an X1E80100 and willing to help debug this issue further if you're interested, although I'll need guidance as I've done very little graphics development.

Alonely0 avatar Dec 30 '24 14:12 Alonely0

I'd first change the shader into something trivia, i.e. vertex doing nothing and fragment outputting a constant color. I assume that pipeline would be created successfully. And then I'd manually bisect the shader code (between the failing point and the successful point) to see which exact construct is making the driver to freak out.

Once we find the construct, we can work with current Naga developers to put a workaround in place in the shading translation.

kvark avatar Jan 03 '25 06:01 kvark

After I changed the fragment part in bunnymark, it doesn't crash, and it renders a white box. Its seems like the issue is textureSample function call

@fragment fn fs_main(vertex: VertexOutput) -> @location(0) vec4<f32> { return vec4<f32>(1.0, 1.0, 1.0, 1.0); }

HgGamer avatar Jan 03 '25 08:01 HgGamer

After a bit of testing it seems like the crash occurs when you try to access vertex properties in fs_main

HgGamer avatar Jan 03 '25 10:01 HgGamer

I can repro @HgGamer's test:

Acessing vertex.position in the shader works, but vertex.color or vertex.tex_coords causes the application to crash.

DeviceInformation {
   is_software_emulated: false,
   device_name: "Qualcomm(R) Adreno(TM) X1-85 GPU"
   driver_name: "Qualcomm Technologies Inc. Adreno Vulkan Driver",
   driver_info: "Driver Build: b1f52b5da7, Ibad1ce48fe, 1723618625\nDate: 08/14/2024\nCompiler Version: E031.47.09.00\nDriver Branch: \n" 
}

andreban avatar Jan 04 '25 10:01 andreban

Also, I tried multiple texture formats because I read somewhere that the GPU doesn't support every format.

HgGamer avatar Jan 04 '25 10:01 HgGamer

There's a workaround implemented in Blender a couple months ago, which could be related and points to a driver issue with optimized SPIR-V. The workaround for Blender was to disable some optimizations (this is beyond my knowledge, so just a guess).

andreban avatar Jan 04 '25 10:01 andreban

Thank you both for the investigation! The Blender thread appears to be very relevant but unfortunately lacking details. It would be useful to read the Qualcomm driver devs take on this. Shaderc optimizations is basically spirv-opt from SPIR-V tools. Blade doesn't use it - we have a pure-Rust shader pipeline, and we aren't running the shader through spirv-opt since we assume the driver already doing this anyway at pipeline creation.

There appear to be multiple other threads about Adreno GPU issues with SPIR-V shaders:

  • https://github.com/devmiyax/yabause/issues/998
  • https://github.com/o3de/o3de-extras/issues/625
  • https://community.khronos.org/t/adreno-506-vkcreatecomputepipelines-has-returned-vk-incomplete/104071/13

kvark avatar Jan 04 '25 17:01 kvark

On snapdragon devices there are two vulkan drivers, one is native, and the other is emulated with DX12 (Dozen). Its maybe possible to test it with each drivers.

HgGamer avatar Jan 23 '25 08:01 HgGamer

Originally posted by @evgri243 in #19739 I assume it is more valuable here.

Just as an intro. I tried all possible combinations yesterday.

Build for: "x86_64-pc-windows-msvc" and "aarch64-pc-windows-msvc", Executed it with "Force Physical Device Only Preset" in Portability Loader Configuration of kvconfig:

  • Qualcomm(R) Adreno (TM) X1-85 GPU with driver: "C:\WINDOWS\System32\DriverStore\FileRepository\qcdx8380.inf_arm64_1b5d57031d8d3198.\qcvkarm64xum.dll": fails with the error above for both build architectures: Result::unwrap() on an Err value: ([0x0], ERROR_UNKNOWN) in blade-graphics\\src\\vulkan\\pipeline.rs:614.
  • Microsoft Direct3D12 (Qualcomm(R) Adreno(TM) X1-85 GPU) with Vulkan 1.2.295 driver: fails with for both architectures with Unable to init GPU context: NoSupportedDeviceFound in crates\gpui\src\platform\windows\platform.rs:111.
  • Microsoft Direct3D12 (Qualcomm(R) Adreno(TM) X1-85 GPU) with Vulkan 1.2.295 driver: fails with for both architectures with Unable to init GPU context: NoSupportedDeviceFound in crates\gpui\src\platform\windows\platform.rs:111.

So far, no working configuration.

Direct3D version: 1.611.1-81528511 DXCore version: 10.0.26100.1-240331-1435.ge-release Windows version: 10.0.26120.3576 Graphics driver: the latest v31.0.96.0 beta OpenCL™, OpenGL®, and Vulkan® Compatibility Pack is installed, hence the Microsoft 3D12 devices. Tested with the latest master by the date Blade with minimal changes for obj2 library compatibility. vkcube works with every forced device.

evgri243 avatar Mar 24 '25 13:03 evgri243

I can reproduce this with building it myself with MSVC 2022 on Windows on ARM and also with a CLANGARM64 build in MSYS2. Both fail in the exact same place:

[2025-04-03T20:50:58-06:00 ERROR client] Element not found. (0x80070490)
[2025-04-03T20:50:58-06:00 ERROR blade_graphics::hal::surface] No composite alpha flag for transparency: OPAQUE
[2025-04-03T20:50:59-06:00 ERROR zed::reliability] {
  "thread": "main",
  "payload": "called `Result::unwrap()` on an `Err` value: ([0x0], ERROR_UNKNOWN)",
  "location_data": {
    "file": "C:\\Users\\Hernan\\.cargo\\git\\checkouts\\blade-26bd638d88226de3\\b16f5c7\\blade-graphics\\src\\vulkan\\pipeline.rs",
    "line": 614
  },
  "backtrace": [
    "backtrace::backtrace::dbghelp64::trace+191492748",
    "backtrace::backtrace::trace_unsynchronized<backtrace::capture::impl$4::create::closure_env$0>+191492748",
    "backtrace::backtrace::trace<backtrace::capture::impl$4::create::closure_env$0>+191492484",
    "backtrace::capture::Backtrace::create+191468952",
    "backtrace::capture::Backtrace::new+191468836",
    "zed::reliability::init_panic_hook::closure$0+1470324",
    "alloc::boxed::impl$30::call+194289040",
    "std::panicking::rust_panic_with_hook+194289040",
    "std::panicking::begin_panic_handler::closure$0+194288284",
    "std::sys::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::closure_env$0,never$>+194280700"
  ],
  "app_version": "0.180.2",
  "app_commit_sha": "3775496b84cfc9ebf27376119742f3484d4997f2",
  "release_channel": "stable",
  "target": "aarch64-pc-windows-msvc",
  "os_name": "Windows",
  "os_version": "10.0.26100",
  "architecture": "aarch64",
  "panicked_on": 1743735059182,
  "system_id": "f70ef1ce-d154-4628-bc79-66887ee17028",
  "installation_id": "ed8b4376-2159-4e7d-bfd1-300540a4f679",
  "session_id": "c901de50-bbf8-4e47-9abc-081f57784185"
}

hmartinez82 avatar Apr 04 '25 03:04 hmartinez82

I'm trying to get zed running on Windows with lavapipe, at least for the time being while the Snapdragon X Vulkan driver is in this broken state, but.

I can't even get the blade-graphics vehicle example to work with lavapipe, but the list of required Vulkan extensions in https://crates.io/crates/blade-graphics does not seem to be comprehensive (see https://github.com/kvark/blade/issues/255), and when I try to run the example I get a KHR_external_memory_win32 not implemented error, so I added this request: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13017 .

Lavapipe already supports ``KHR_external_memory_fd` so blade-graphics seem to work on Linux with lavapipe.

hmartinez82 avatar Apr 21 '25 23:04 hmartinez82

Update: lavapipe now works, see #255 As for the actual issue with Snapdragon, we need to find a way to get more feedback from the driver, if possible. Generally for this configuration, I'd expect it to be an internal failure of the driver not behaving up to specification.

kvark avatar Jun 02 '25 05:06 kvark

@kvark Do you mean zed with #255 now works? I still have to give it a try.

hmartinez82 avatar Jun 02 '25 05:06 hmartinez82