open-gpu-kernel-modules icon indicating copy to clipboard operation
open-gpu-kernel-modules copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

Suspend doesn't work when PreserveVideoMemoryAllocations is enabled

Open Monsterovich opened this issue 2 years ago • 197 comments

NVIDIA Open GPU Kernel Modules Version

525.85.05

Does this happen with the proprietary driver (of the same version) as well?

Yes

Operating System and Version

Linux Mint 21.1

Kernel Release

Linux pc 5.19.15-xanmod1 #0~20221012.git85103f7 SMP Wed Oct 12 15:04:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Hardware: GPU

GPU 0: NVIDIA GeForce RTX 3060 Ti (UUID: GPU-039a34f7-087c-dc8f-a607-e0d937443d0f)

Describe the bug

An error occurs when trying to boot into the system after suspend if PreserveVideoMemoryAllocations is enabled:

[   10.255125] NVRM: GPU 0000:05:00.0: PreserveVideoMemoryAllocations module parameter is set. System Power Management attempted without driver procfs suspend interface. Please refer to the 'Configuring Power Management Support' section in the driver README.
[   10.255127] nvidia 0000:05:00.0: PM: pci_pm_freeze(): nv_pmops_freeze+0x0/0x20 [nvidia] returns -5
[   10.255200] nvidia 0000:05:00.0: PM: dpm_run_callback(): pci_pm_freeze+0x0/0xc0 returns -5
[   10.255203] nvidia 0000:05:00.0: PM: failed to quiesce async: error -5

To Reproduce

  1. Create a file /etc/modprobe.d/nvidia-power-management.conf with the following content:
options nvidia NVreg_PreserveVideoMemoryAllocations=1 NVreg_TemporaryFilePath=/var/tmp
  1. sudo update-initramfs -c -k all to re-generate the kernels.
  2. Reboot.
  3. Go to suspend mode.
  4. Get an error when trying to load a saved state.

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

https://forums.developer.nvidia.com/t/cannot-suspend-gtx3060-laptop/244143/9 https://forums.developer.nvidia.com/t/preservevideomemoryallocations-systemd-services-causes-resume-from-hibernate-to-fail/233643/3

Monsterovich avatar Mar 11 '23 19:03 Monsterovich

Same bug with suspend mode on Arch Linux, when I suspend I get a black screen. Running:

linux 6.2.2.arch2-1
nvidia-open 525.89.02-7
xf86-video-nouveau 1.0.17-2

David7ce avatar Mar 12 '23 00:03 David7ce

How did you trigger suspend on this system? It looks like the relevant systemd units are enabled but I don't see any evidence that they were triggered. Something needs to run /usr/bin/nvidia-sleep.sh suspend before suspend and /usr/bin/nvidia-sleep.sh resume after resume.

For context, the video memory preservation process needs to happen before the kernel's low-level suspend process is triggered. The message from the driver implies that something bypassed "System Power Management attempted without driver procfs suspend interface. Please refer to the 'Configuring Power Management Support' section in the driver README" implies that that bracketing didn't happen for some reason.

aaronp24 avatar Mar 13 '23 19:03 aaronp24

With NVIDiA propietary graphics suspend mode and wake up (with NVIDIA RTX 2060 Super or Turing) works fine. And I suspend the systems with KDE GUI or via terminal typing systemctl suspend. And the problem was when I tried to wake up the system, always black screen...

David7ce avatar Mar 14 '23 00:03 David7ce

How did you trigger suspend on this system? It looks like the relevant systemd units are enabled but I don't see any evidence that they were triggered.

Nevertheless, all these services are enabled in my system, but they don't work as they should.

systemctl list-unit-files | grep nvidia
nvidia-hibernate.service                                                  enabled         enabled
nvidia-persistenced.service                                               static          -
nvidia-resume.service                                                     enabled         enabled
nvidia-suspend.service                                                    enabled         enabled

Monsterovich avatar Mar 17 '23 21:03 Monsterovich

Hi All, Please test with latest released driver 535.43.02 and share test results.

amrit1711 avatar Jun 05 '23 17:06 amrit1711

still doesn't work with 535.43.02, though works fine on proprietary module

Sid127 avatar Jun 15 '23 17:06 Sid127

Hi All, Please test with latest released driver 535.43.02 and share test results.

@amrit1711

NVidia driver 535.54.03.

Regular suspend now doesn't work on open-source driver (works on proprietary module just as @Sid127 has mentioned). The screen simply turns off after waking up.

dmesg.log

июл 01 22:08:37 pc kernel: NVRM rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00000; hParent=0x00010001; hObject=0x00010011; hClass=0x0000c670; paramsSize=0x00000000; paramsStatus=0x00000062; status=0x00000062
июл 01 22:08:37 pc kernel: nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])
июл 01 22:08:37 pc kernel: NVRM serverFreeResourceTree: hObject 0x10011 not found for client 0xc1d00000
июл 01 22:08:37 pc kernel: NVRM unixCallVideoBIOS: int10h(4f02, 0177) vesa call failed! (02a2, 0000)
июл 01 22:08:37 pc kernel: NVRM rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00000; hParent=0x00010001; hObject=0x00010011; hClass=0x0000c670; paramsSize=0x00000000; paramsStatus=0x00000062; status=0x00000062
июл 01 22:08:37 pc kernel: nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])
июл 01 22:08:37 pc kernel: NVRM serverFreeResourceTree: hObject 0x10011 not found for client 0xc1d00000
июл 01 22:08:37 pc kernel: NVRM unixCallVideoBIOS: int10h(4f02, 0177) vesa call failed! (02a2, 0000)
июл 01 22:08:37 pc kernel: NVRM rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00000; hParent=0x00010001; hObject=0x00010011; hClass=0x0000c670; paramsSize=0x00000000; paramsStatus=0x00000062; status=0x00000062
июл 01 22:08:37 pc kernel: nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])
июл 01 22:08:37 pc kernel: NVRM serverFreeResourceTree: hObject 0x10011 not found for client 0xc1d00000
июл 01 22:08:37 pc kernel: NVRM unixCallVideoBIOS: int10h(4f02, 0177) vesa call failed! (02a2, 0000)
июл 01 22:08:37 pc kernel: NVRM rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00000; hParent=0x00010001; hObject=0x00010011; hClass=0x0000c670; paramsSize=0x00000000; paramsStatus=0x00000062; status=0x00000062
июл 01 22:08:37 pc kernel: nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])

Suspend to disk doesn't work either. The same message as in the first post.

I also see garbage messages in dmesg:

[   12.759538] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 11 shares a pVAS with channel 12
[   12.760963] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 12 shares a pVAS with channel 13
[   12.762454] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 13 shares a pVAS with channel 14
[   12.763909] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 14 shares a pVAS with channel 15
[   12.765581] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 15 shares a pVAS with channel 16
[   12.767228] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 16 shares a pVAS with channel 17
[   12.768622] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 17 shares a pVAS with channel 18
[   12.920867] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 11 shares a pVAS with channel 12
[   12.922407] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 12 shares a pVAS with channel 13
[   12.923962] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 13 shares a pVAS with channel 14
[   12.925519] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 14 shares a pVAS with channel 15
[   12.926825] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 15 shares a pVAS with channel 16
[   12.928098] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 16 shares a pVAS with channel 17
[   12.929377] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 17 shares a pVAS with channel 18
[   13.608901] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 11 shares a pVAS with channel 12
[   13.611364] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 12 shares a pVAS with channel 13
[   13.613183] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 13 shares a pVAS with channel 14
[   13.614439] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 14 shares a pVAS with channel 15
[   13.616568] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 15 shares a pVAS with channel 16
[   13.620816] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 16 shares a pVAS with channel 17
[   13.622490] NVRM kgrctxUnmapAssociatedCtxBuffers_IMPL: TSG 9 Channel 17 shares a pVAS with channel 18

and

[    5.099933] NVRM cpuidInfoAMD: Unrecognized AMD processor in cpuidInfoAMD

But I guess they don't affect anything.

Monsterovich avatar Jul 01 '23 20:07 Monsterovich

Same problem on GTX 3080 with kernel 6.4.2 and open driver 535.54.03.

During suspend preparation, after nvidia-suspend.service has been activated kernel complains:

kernel: NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mem_list.c:293kernel: NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mem_list.c:293
kernel: NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from pRmApi->Alloc(pRmApi, pMemoryManager->hClient>
kernel: NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from memdescSendMemDescToGSP(pGpu, pFbsr->pSysMemD>
kernel: NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from _fbsrInitGsp(pGpu, pFbsr) @ fbsr_gm107.c:548

Looks like some sort of GSP firmware and driver incompatibility/miscommunication.

zaltysz avatar Jul 06 '23 19:07 zaltysz

I'm also has this issue in

  • Ubuntu 23.04
  • NVIDIA Corporation TU116 [GeForce GTX 1660 SUPER] [10de:21c4]
  • Kernel 6.2.0-24-generic
  • Nvidia 525.116.04 / 535.54.03

hugh712 avatar Jul 10 '23 12:07 hugh712

Suspend leads to similar error logs before suspend and a kernel panic while waking up from suspend on my system. It is a laptop with an AMD Ryzen 7 5800H and an RTX 3070 Mobile running kernel 6.3.9 and version 535.54.03. I boot with the following flags: nvidia-drm.modeset=1 NVreg_EnableResizableBar=1 nvidia.NVreg_PreserveVideoMemoryAllocations=1 nvidia.NVreg_OpenRmEnableUnsupportedGpus=1

Before suspend I find the following in the logs:

NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mem_list.c:293
NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from pRmApi->Alloc(pRmApi, pMemoryManager->hClient, pMemoryManager->
NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from memdescSendMemDescToGSP(pGpu, pFbsr->pSysMemDesc, &hSysMem) @ >
NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from _fbsrInitGsp(pGpu, pFbsr) @ fbsr_gm107.c:548

After suspend I get more error logs and a kernel panic:

NVRM: GPU at PCI:0000:01:00: GPU-cd48b96b-c536-1a29-44c0-809f5b2531e4
NVRM: Xid (PCI:0000:01:00): 45, pid=975, name=(udev-worker), Ch 00000000
NVRM: Xid (PCI:0000:01:00): 45, pid=975, name=(udev-worker), Ch 00000001
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000002
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000003
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000004
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000005
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000006
NVRM: Xid (PCI:0000:01:00): 45, pid=3823, name=Renderer, Ch 00000007
NVRM: Xid (PCI:0000:01:00): 45, pid=2819, name=sway, Ch 00000008
NVRM: Xid (PCI:0000:01:00): 45, pid=2819, name=sway, Ch 00000009
NVRM: Xid (PCI:0000:01:00): 45, pid=2819, name=sway, Ch 0000000a
NVRM: Xid (PCI:0000:01:00): 45, pid=2905, name=Xwayland, Ch 0000000b
NVRM: Xid (PCI:0000:01:00): 45, pid=2905, name=Xwayland, Ch 0000000c
ata2: SATA link down (SStatus 0 SControl 300)
NVRM: Xid (PCI:0000:01:00): 45, pid=2905, name=Xwayland, Ch 0000000d
NVRM: Xid (PCI:0000:01:00): 45, pid=23536, name=slack, Ch 0000000e
NVRM: Xid (PCI:0000:01:00): 45, pid=23536, name=slack, Ch 0000000f
NVRM: Xid (PCI:0000:01:00): 45, pid=23536, name=slack, Ch 00000010
NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70e000 returned garbage 0x0
NVRM nvAssertOkFailedNoLog: Assertion failed: Generic memory error [NV_ERR_MEMORY_ERROR] (0x00000072) returned from kbusVerifyBar2_HAL(pGpu, pKernelBus, NULL, NULL, 0, 0) @ kern_bus_gm107.c:457
NVRM nvAssertOkFailedNoLog: Assertion failed: Generic memory error [NV_ERR_MEMORY_ERROR] (0x00000072) returned from gpuStateLoad(pGpu, IS_GPU_GC6_STATE_EXITING(pGpu) ? GPU_STATE_FLAGS_PRESERVING | GPU_STATE_FL>
nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])
NVRM serverFreeResourceTree: hObject 0x10011 not found for client 0xc1d00000
NVRM rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00000; hParent=0x00010001; hObject=0x00010011; hClass=0x0000c670; paramsSize=0x00000000; paramsStatus=0x00000062; status=0x00000062
nvidia-modeset: ERROR: GPU:0: Failed to initialize display engine: 0x62 (Reset required [NV_ERR_RESET_REQUIRED])
NVRM serverFreeResourceTree: hObject 0x10011 not found for client 0xc1d00000
BUG: kernel NULL pointer dereference, address: 000000000000026c
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 15dde0067 P4D 15dde0067 PUD 1270b5067 PMD 0 
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 3 PID: 2819 Comm: sway Tainted: G           O       6.3.9 #1-NixOS
Hardware name: LENOVO 82N6/LNVNB161216, BIOS GKCN60WW 03/07/2023
RIP: 0010:nvAssignDefaultUsageBounds+0x94/0x1a0 [nvidia_modeset]
Code: 43 0c 00 89 43 08 49 8b 84 24 60 0a 00 00 48 8b 80 98 01 00 00 48 85 c0 74 b4 4c 89 e7 e8 24 1a 8f ef ba 04 00 00 00 48 63 ca <8b> 74 c8 08 85 f6 74 5c 48 8d 04 c8 0f b7 48 08 66 89 4b 04 0f b7
RSP: 0018:ffffaa0c088a74c0 EFLAGS: 00010206
RAX: 0000000000000244 RBX: ffff9b5c26c5ea58 RCX: 0000000000000004
RDX: 0000000000000004 RSI: ffff9b5c26c5ea40 RDI: ffff9b5c3a8e0008
RBP: ffff9b5c3a8e06a0 R08: 0000000000000005 R09: 0000000000000000
R10: 00000000000006c4 R11: 0000000000000000 R12: ffff9b5c3a8e0008
R13: ffff9b5c26c5ea40 R14: ffff9b5c26c5ea58 R15: ffff9b5c3a8e07e0
FS:  00007ff08edc6f80(0000) GS:ffff9b630dcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000000026c CR3: 0000000152924000 CR4: 0000000000750ee0
PKRU: 55555554
Call Trace:
 <TASK>
 ? __die+0x23/0x70
 ? page_fault_oops+0x17d/0x4b0
 ? number+0x34a/0x3e0
 ? exc_page_fault+0x6a/0x150
 ? asm_exc_page_fault+0x26/0x30
 ? nvAssignDefaultUsageBounds+0x94/0x1a0 [nvidia_modeset]
 ConstructHwModeTimingsViewPort.constprop.0+0x118/0x160 [nvidia_modeset]
 nvConstructHwModeTimingsEvo+0x344/0x5e0 [nvidia_modeset]
 ? vsnprintf+0x2cb/0x550
 ? nvEvoLogInfoString+0xa0/0xc0 [nvidia_modeset]
 ValidateMode+0x1bb/0x700 [nvidia_modeset]
 ? ValidateMode+0x14f/0x700 [nvidia_modeset]
 nvValidateModeIndex+0x3f8/0x5d0 [nvidia_modeset]
 ? nvValidateModeIndex+0x3ca/0x5d0 [nvidia_modeset]
 ? GetDispAttributeValidValues+0x60/0x60 [nvidia_modeset]
 ValidateModeIndex+0x59/0x70 [nvidia_modeset]
 nvKmsIoctl+0xde/0x220 [nvidia_modeset]
 nvkms_ioctl_from_kapi+0x6b/0xc0 [nvidia_modeset]
 GetDisplayMode+0x71/0x210 [nvidia_modeset]
 nv_drm_connector_get_modes+0x9e/0x150 [nvidia_drm]
 drm_helper_probe_single_connector_modes+0x1a0/0x530 [drm_kms_helper]
 ? drm_atomic_get_property+0xff/0x5f0 [drm]
 drm_mode_getconnector+0x394/0x4a0 [drm]
 ? drm_mode_obj_get_properties_ioctl+0x14a/0x1b0 [drm]
 ? __pfx_drm_mode_getconnector+0x10/0x10 [drm]
 drm_ioctl_kernel+0xcd/0x170 [drm]
 drm_ioctl+0x267/0x4a0 [drm]
 ? __pfx_drm_mode_getconnector+0x10/0x10 [drm]
 __x64_sys_ioctl+0x94/0xd0
 do_syscall_64+0x3e/0x90
 entry_SYSCALL_64_after_hwframe+0x72/0xdc
RIP: 0033:0x7ff08fe3529f
Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 18 48 8b 44 24 18 64 48 2b 04 25 28 00 00
RSP: 002b:00007ffc649f08d0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000001a42690 RCX: 00007ff08fe3529f
RDX: 00007ffc649f0970 RSI: 00000000c05064a7 RDI: 000000000000000b
RBP: 00007ffc649f0970 R08: 0000000000000007 R09: 0000000001693b60
R10: 7909572cf32bdd3d R11: 0000000000000246 R12: 00000000c05064a7
R13: 000000000000000b R14: 00000000c05064a7 R15: 00007ffc649f0970
 </TASK>
Modules linked in: uhid uvcvideo videobuf2_vmalloc uvc videobuf2_memops videobuf2_v4l2 snd_usb_audio videodev snd_usbmidi_lib videobuf2_common snd_rawmidi mc af_packet ctr snd_seq_dummy snd_hrtimer snd_seq snd>
 cfg80211 snd_timer snd_soc_cs35l41_lib ucsi_acpi crc32_pclmul realtek sp5100_tco snd polyval_clmulni ideapad_laptop(O) wdat_wdt typec_ucsi polyval_generic gf128mul cm32181 tpm_crb sparse_keymap mdio_devres nv>
 crct10dif_pclmul crct10dif_common rtc_cmos serio dm_mod dax btrfs blake2b_generic xor libcrc32c crc32c_generic crc32c_intel raid6_pq amdgpu i2c_algo_bit drm_ttm_helper ttm agpgart video wmi iommu_v2 drm_buddy>
CR2: 000000000000026c
---[ end trace 0000000000000000 ]---
RIP: 0010:nvAssignDefaultUsageBounds+0x94/0x1a0 [nvidia_modeset]
Code: 43 0c 00 89 43 08 49 8b 84 24 60 0a 00 00 48 8b 80 98 01 00 00 48 85 c0 74 b4 4c 89 e7 e8 24 1a 8f ef ba 04 00 00 00 48 63 ca <8b> 74 c8 08 85 f6 74 5c 48 8d 04 c8 0f b7 48 08 66 89 4b 04 0f b7
RSP: 0018:ffffaa0c088a74c0 EFLAGS: 00010206
RAX: 0000000000000244 RBX: ffff9b5c26c5ea58 RCX: 0000000000000004
RDX: 0000000000000004 RSI: ffff9b5c26c5ea40 RDI: ffff9b5c3a8e0008
RBP: ffff9b5c3a8e06a0 R08: 0000000000000005 R09: 0000000000000000
R10: 00000000000006c4 R11: 0000000000000000 R12: ffff9b5c3a8e0008
R13: ffff9b5c26c5ea40 R14: ffff9b5c26c5ea58 R15: ffff9b5c3a8e07e0
FS:  00007ff08edc6f80(0000) GS:ffff9b630dcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000000026c CR3: 0000000152924000 CR4: 0000000000750ee0
PKRU: 55555554

FlorianFranzen avatar Jul 13 '23 20:07 FlorianFranzen

Thank you for the reports and logs. We understand the issue, and NVIDIA internal bug 4193761 tracks this.

Unfortunately it will probably be the release cycle after 535.xx, at the earliest, before NVreg_PreserveVideoMemoryAllocations works correctly with the open-gpu-kernel-modules. Sorry, and thanks for your patience.

aritger avatar Jul 13 '23 21:07 aritger

Is there any progress?

Longhao-Chen avatar Sep 29 '23 11:09 Longhao-Chen

Any updates?

unkindypie avatar Oct 18 '23 17:10 unkindypie

Still doesn't work with nvidia-open 545.29.02

I'm stuck between a rock and a hard place right now as nvidia proprietary driver throws ACPI errors on my system and causes general instability. nvidia-open works fine except sleep and hibernate which don't work at all. So I have to choose between a stable system and suspend support, 3 whole years after the release of my GPU (RTX 3080). Very frustrated with nvidia right now.

ifaigios avatar Nov 08 '23 23:11 ifaigios

Found solution here! See https://forums.developer.nvidia.com/t/cannot-suspend-gtx3060-laptop/244143/12

  1. Download the nvidia-services-install.zip
  2. cd ~/Downloads/ then extract sudo unzip nvidia-services-install.zip -d /usr/share/doc/nvidia-driver-545/ (change the driver version number to yours)
  3. Modify the file sudo vim setup-nvdia-suspend.sh, change this line to TMPL_PATH=/usr/share/doc/nvidia-driver-545, save and quit
  4. Then run sudo ./setup-nvdia-suspend.sh
  5. Try suspend, it should works

MXXXXXS avatar Nov 09 '23 12:11 MXXXXXS

Found solution here! See https://forums.developer.nvidia.com/t/cannot-suspend-gtx3060-laptop/244143/12

1. Download the [nvidia-services-install.zip](https://forums.developer.nvidia.com/uploads/short-url/2ZtALqhLyZlrEx0a5ulzFsc3CXD.zip)

2. `cd ~/Downloads/` then extract `sudo unzip nvidia-services-install.zip -d /usr/share/doc/nvidia-driver-545/` (change the driver version number to yours)

3. Modify the file `sudo vim setup-nvdia-suspend.sh`, change this line to
   `TMPL_PATH=/usr/share/doc/nvidia-driver-545`, save and quit

4. Then run `sudo ./setup-nvdia-suspend.sh`

5. Try suspend, it should works

I think this is with the proprietary kernel modules and not the open kernel modules?

veldenb avatar Nov 09 '23 14:11 veldenb

Found solution here! See https://forums.developer.nvidia.com/t/cannot-suspend-gtx3060-laptop/244143/12

1. Download the [nvidia-services-install.zip](https://forums.developer.nvidia.com/uploads/short-url/2ZtALqhLyZlrEx0a5ulzFsc3CXD.zip)

2. `cd ~/Downloads/` then extract `sudo unzip nvidia-services-install.zip -d /usr/share/doc/nvidia-driver-545/` (change the driver version number to yours)

3. Modify the file `sudo vim setup-nvdia-suspend.sh`, change this line to
   `TMPL_PATH=/usr/share/doc/nvidia-driver-545`, save and quit

4. Then run `sudo ./setup-nvdia-suspend.sh`

5. Try suspend, it should works

I think this is with the proprietary kernel modules and not the open kernel modules?

Oh... overlooked the title, yes it works on the proprietary driver, not tested on the open driver

MXXXXXS avatar Nov 09 '23 14:11 MXXXXXS

Found solution here! See https://forums.developer.nvidia.com/t/cannot-suspend-gtx3060-laptop/244143/12

1. Download the [nvidia-services-install.zip](https://forums.developer.nvidia.com/uploads/short-url/2ZtALqhLyZlrEx0a5ulzFsc3CXD.zip)

2. `cd ~/Downloads/` then extract `sudo unzip nvidia-services-install.zip -d /usr/share/doc/nvidia-driver-545/` (change the driver version number to yours)

3. Modify the file `sudo vim setup-nvdia-suspend.sh`, change this line to
   `TMPL_PATH=/usr/share/doc/nvidia-driver-545`, save and quit

4. Then run `sudo ./setup-nvdia-suspend.sh`

5. Try suspend, it should works

This has not fixed the issue for me, on the proprietary driver or the open driver.

edwloef avatar Nov 17 '23 10:11 edwloef

In my case (Nvidia driver 545.29.06) resume sometimes works, sometimes not. nvidia-hibernate.service, nvidia-resume.service and nvidia-suspend.service are enabled.

"dmesg" shows one failed resume action:

$ dmesg -T | grep PM:
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x0290f000-0x0290ffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x0291e000-0x0291efff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x02928000-0x02928fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x02947000-0x02947fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x02948000-0x02948fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0x02950000-0x02950fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbc4d000-0xbbc4ffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbc50000-0xbbc94fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbc95000-0xbbd08fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbd0b000-0xbbe10fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe12000-0xbbe12fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe13000-0xbbe1afff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe1b000-0xbbe1bfff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe1c000-0xbbe1dfff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe1e000-0xbbe25fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe26000-0xbbe46fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbbe47000-0xbbe89fff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbc000000-0xbf7fffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xbf800000-0xdfffffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xe0000000-0xe3ffffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xe4000000-0xfed1bfff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xfed20000-0xfeffffff]
[Thu Dec  7 08:41:28 2023] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff]
[Thu Dec  7 08:41:28 2023] ACPI: PM: Registering ACPI NVS region [mem 0xbbc50000-0xbbc94fff] (282624 bytes)
[Thu Dec  7 08:41:28 2023] ACPI: PM: Registering ACPI NVS region [mem 0xbbe12000-0xbbe12fff] (4096 bytes)
[Thu Dec  7 08:41:28 2023] ACPI: PM: Registering ACPI NVS region [mem 0xbbe1b000-0xbbe1bfff] (4096 bytes)
[Thu Dec  7 08:41:28 2023] ACPI: PM: Registering ACPI NVS region [mem 0xbbe1e000-0xbbe25fff] (32768 bytes)
[Thu Dec  7 08:41:28 2023] ACPI: PM: Registering ACPI NVS region [mem 0xbbe47000-0xbbe89fff] (274432 bytes)
[Thu Dec  7 08:41:28 2023] PM: RTC time: 07:41:28, date: 2023-12-07
[Thu Dec  7 08:41:28 2023] ACPI: PM: (supports S0 S1 S3 S4 S5)
[Thu Dec  7 08:41:29 2023] PM:   Magic number: 3:436:672
[Thu Dec  7 08:41:32 2023] PM: Image signature found, resuming
[Thu Dec  7 08:41:32 2023] PM: hibernation: resume from hibernation
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x00000000-0x00000fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x000a0000-0x000fffff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x0290f000-0x0290ffff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x0291e000-0x0291efff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x02928000-0x02928fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x02947000-0x02948fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0x02950000-0x02950fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0xbbc4d000-0xbbd08fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0xbbd0b000-0xbbe10fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0xbbe12000-0xbbe89fff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Marking nosave pages: [mem 0xbc000000-0xffffffff]
[Thu Dec  7 08:41:33 2023] PM: hibernation: Basic memory bitmaps created
[Thu Dec  7 08:41:33 2023] PM: Using 3 thread(s) for decompression
[Thu Dec  7 08:41:33 2023] PM: Loading and decompressing image data (1504949 pages)...
[Thu Dec  7 08:41:33 2023] PM: Image loading progress:   0%
[Thu Dec  7 08:41:36 2023] PM: Image loading progress:  10%
[Thu Dec  7 08:41:37 2023] PM: Image loading progress:  20%
[Thu Dec  7 08:41:39 2023] PM: Image loading progress:  30%
[Thu Dec  7 08:41:40 2023] PM: Image loading progress:  40%
[Thu Dec  7 08:41:41 2023] PM: Image loading progress:  50%
[Thu Dec  7 08:41:43 2023] PM: Image loading progress:  60%
[Thu Dec  7 08:41:44 2023] PM: Image loading progress:  70%
[Thu Dec  7 08:41:45 2023] PM: Image loading progress:  80%
[Thu Dec  7 08:41:47 2023] PM: Image loading progress:  90%
[Thu Dec  7 08:41:48 2023] PM: Image loading progress: 100%
[Thu Dec  7 08:41:48 2023] PM: Image loading done
[Thu Dec  7 08:41:48 2023] PM: hibernation: Read 6019796 kbytes in 14.98 seconds (401.85 MB/s)
[Thu Dec  7 08:41:48 2023] PM: Image successfully loaded
[Thu Dec  7 08:41:48 2023] nvidia 0000:01:00.0: PM: pci_pm_freeze(): nv_pmops_freeze+0x0/0x20 [nvidia] returns -5
[Thu Dec  7 08:41:48 2023] nvidia 0000:01:00.0: PM: dpm_run_callback(): pci_pm_freeze+0x0/0xc0 returns -5
[Thu Dec  7 08:41:48 2023] nvidia 0000:01:00.0: PM: failed to quiesce async: error -5
[Thu Dec  7 08:41:48 2023] PM: hibernation: Failed to load image, recovering.
[Thu Dec  7 08:41:49 2023] PM: hibernation: Basic memory bitmaps freed
[Thu Dec  7 08:41:49 2023] PM: hibernation: resume failed (-5)

bjoernv avatar Dec 07 '23 11:12 bjoernv

Suspend seems to work sort of, but when waking up it fails by not turning the display on. The machine is sill responsible. While going into sleep mode I get error messages starting with:

[   61.128594] NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mem_list.c:268
[   61.128750] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from pRmApi->Alloc(pRmApi, pMemoryManager->hClient, pMemoryManager->hSubdevice, pHandle, hClass, &listAllocParams, sizeof(listAllocParams)) @ mem_desc.c:4841
[   61.128788] NVRM serverFreeResourceTree: hObject 0xcaf00003 not found for client 0xc1e00002
[   61.128803] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from memdescSendMemDescToGSP(pGpu, pFbsr->pSysMemDesc, &hSysMem) @ fbsr_gm107.c:113
[   61.128830] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from _fbsrInitGsp(pGpu, pFbsr) @ fbsr_gm107.c:550

And when waking up:

[   67.529695] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
[   67.530492] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
[   67.530999] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
[   67.531515] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0

(full log below)

Specs

Linux desktop 6.6.6-060606-generic #202312111032 SMP PREEMPT_DYNAMIC Mon Dec 11 10:41:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
nvidia-driver-545-open 545.29.06-0ubuntu0~gpu23.10.1

$ systemctl list-unit-files | grep nvidia
nvidia-hibernate.service                                           enabled         enabled
nvidia-persistenced.service                                        static          -
nvidia-resume.service                                              enabled         enabled
nvidia-suspend.service                                             enabled         enabled

$ nvidia-smi 
Mon Dec 11 22:37:59 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.29.06              Driver Version: 545.29.06    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3080        Off | 00000000:0B:00.0  On |                  N/A |
|  0%   43C    P8              23W / 320W |    498MiB / 10240MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
Syslog with dmesg output 
2023-12-11T22:19:47.492445+01:00 desktop systemd[1]: Reached target sleep.target - Sleep.
2023-12-11T22:19:47.509928+01:00 desktop systemd[1]: Starting nvidia-suspend.service - NVIDIA system suspend actions...
2023-12-11T22:19:47.526542+01:00 desktop suspend: nvidia-suspend.service
2023-12-11T22:19:47.526641+01:00 desktop logger[3676]: <13>Dec 11 22:19:47 suspend: nvidia-suspend.service
2023-12-11T22:19:47.780492+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/ldac
2023-12-11T22:19:47.780556+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/aptx_hd
2023-12-11T22:19:47.780703+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx_hd
2023-12-11T22:19:47.780876+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/aptx
2023-12-11T22:19:47.781048+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx
2023-12-11T22:19:47.781223+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/sbc
2023-12-11T22:19:47.781397+01:00 desktop kernel: [   60.990030] rfkill: input handler enabled
2023-12-11T22:19:47.781448+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/sbc
2023-12-11T22:19:47.781575+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/sbc_xq
2023-12-11T22:19:47.781756+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/sbc_xq
2023-12-11T22:19:47.781943+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx_ll_1
2023-12-11T22:19:47.782130+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx_ll_0
2023-12-11T22:19:47.782350+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_1
2023-12-11T22:19:47.782552+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_0
2023-12-11T22:19:47.782749+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/faststream
2023-12-11T22:19:47.782941+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/faststream_duplex
2023-12-11T22:19:47.783120+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/opus_05
2023-12-11T22:19:47.783300+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/opus_05
2023-12-11T22:19:47.783496+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSink/opus_05_duplex
2023-12-11T22:19:47.783675+01:00 desktop bluetoothd[1461]: Endpoint unregistered: sender=:1.42 path=/MediaEndpoint/A2DPSource/opus_05_duplex
2023-12-11T22:19:47.921394+01:00 desktop kernel: [   61.128594] NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mem_list.c:268
2023-12-11T22:19:47.921401+01:00 desktop kernel: [   61.128750] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from pRmApi->Alloc(pRmApi, pMemoryManager->hClient, pMemoryManager->hSubdevice, pHandle, hClass, &listAllocParams, sizeof(listAllocParams)) @ mem_desc.c:4841
2023-12-11T22:19:47.921405+01:00 desktop kernel: [   61.128788] NVRM serverFreeResourceTree: hObject 0xcaf00003 not found for client 0xc1e00002
2023-12-11T22:19:47.921406+01:00 desktop kernel: [   61.128803] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from memdescSendMemDescToGSP(pGpu, pFbsr->pSysMemDesc, &hSysMem) @ fbsr_gm107.c:113
2023-12-11T22:19:47.921406+01:00 desktop kernel: [   61.128830] NVRM nvAssertOkFailedNoLog: Assertion failed: Call not supported [NV_ERR_NOT_SUPPORTED] (0x00000056) returned from _fbsrInitGsp(pGpu, pFbsr) @ fbsr_gm107.c:550
2023-12-11T22:19:48.188871+01:00 desktop systemd[1]: nvidia-suspend.service: Deactivated successfully.
2023-12-11T22:19:48.188938+01:00 desktop systemd[1]: Finished nvidia-suspend.service - NVIDIA system suspend actions.
2023-12-11T22:19:48.189405+01:00 desktop kernel: [   61.398644] NVRM nvCheckFailedNoLog: Check failed: pMemDesc->_pInternalMapping != NULL @ mem_desc.c:2212
2023-12-11T22:19:48.189410+01:00 desktop kernel: [   61.398678] NVRM nvAssertFailedNoLog: Assertion failed: (pPtr = memdescMapInternal(pGpu, pMemDesc, flags)) != NULL @ mem_utils.c:1206
2023-12-11T22:19:48.189412+01:00 desktop kernel: [   61.398712] NVRM nvAssertFailedNoLog: Assertion failed: pEntries != NULL @ gmmu_walk.c:852
2023-12-11T22:19:48.189412+01:00 desktop kernel: [   61.398736] NVRM nvAssertFailedNoLog: Assertion failed: progress == indexHi_tmp - indexLo_tmp + 1 @ mmu_walk.c:1303
2023-12-11T22:19:48.189419+01:00 desktop kernel: [   61.398764] NVRM nvAssertFailedNoLog: Assertion failed: NV_OK == status @ mmu_walk.c:489
2023-12-11T22:19:48.189420+01:00 desktop kernel: [   61.398787] NVRM mmuWalkSparsify: Failed to sparsify VA Range 0x1730000 to 0x176ffff. Status = 0x00000040
2023-12-11T22:19:48.189420+01:00 desktop kernel: [   61.398812] NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mmu_walk_sparse.c:84
2023-12-11T22:19:48.189421+01:00 desktop kernel: [   61.398834] NVRM nvCheckFailedNoLog: Check failed: pMemDesc->_pInternalMapping != NULL @ mem_desc.c:2212
2023-12-11T22:19:48.189421+01:00 desktop kernel: [   61.398859] NVRM nvAssertFailedNoLog: Assertion failed: (pPtr = memdescMapInternal(pGpu, pMemDesc, flags)) != NULL @ mem_utils.c:1206
2023-12-11T22:19:48.189422+01:00 desktop kernel: [   61.398891] NVRM nvAssertFailedNoLog: Assertion failed: pEntries != NULL @ gmmu_walk.c:852
2023-12-11T22:19:48.189422+01:00 desktop kernel: [   61.398914] NVRM nvAssertFailedNoLog: Assertion failed: progress == indexHi_tmp - indexLo_tmp + 1 @ mmu_walk.c:1303
2023-12-11T22:19:48.189423+01:00 desktop kernel: [   61.398942] NVRM nvAssertFailedNoLog: Assertion failed: NV_OK == status @ mmu_walk.c:489
2023-12-11T22:19:48.189425+01:00 desktop kernel: [   61.398964] NVRM mmuWalkUnmap: Failed to unmap VA Range 0x1730000 to 0x176ffff. Status = 0x00000040
2023-12-11T22:19:48.189427+01:00 desktop kernel: [   61.398988] NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mmu_walk_unmap.c:72
2023-12-11T22:19:48.189427+01:00 desktop kernel: [   61.399009] NVRM mmuWalkSparsify: Unmap failed with status = 0x00000040
2023-12-11T22:19:48.189428+01:00 desktop kernel: [   61.399027] NVRM nvAssertFailedNoLog: Assertion failed: NV_OK == unmapStatus @ mmu_walk_sparse.c:95
2023-12-11T22:19:48.189428+01:00 desktop kernel: [   61.399052] NVRM nvCheckFailedNoLog: Check failed: pMemDesc->_pInternalMapping != NULL @ mem_desc.c:2212
2023-12-11T22:19:48.189429+01:00 desktop kernel: [   61.399078] NVRM nvAssertFailedNoLog: Assertion failed: (pPtr = memdescMapInternal(pGpu, pMemDesc, flags)) != NULL @ mem_utils.c:1206
2023-12-11T22:19:48.189430+01:00 desktop kernel: [   61.399109] NVRM nvAssertFailedNoLog: Assertion failed: pEntries != NULL @ gmmu_walk.c:852
2023-12-11T22:19:48.189430+01:00 desktop kernel: [   61.399132] NVRM nvAssertFailedNoLog: Assertion failed: progress == indexHi_tmp - indexLo_tmp + 1 @ mmu_walk.c:1303
2023-12-11T22:19:48.189431+01:00 desktop kernel: [   61.399160] NVRM nvAssertFailedNoLog: Assertion failed: NV_OK == status @ mmu_walk.c:489
2023-12-11T22:19:48.189432+01:00 desktop kernel: [   61.399182] NVRM mmuWalkUnmap: Failed to unmap VA Range 0x1730000 to 0x176ffff. Status = 0x00000040
2023-12-11T22:19:48.189432+01:00 desktop kernel: [   61.399206] NVRM nvAssertFailedNoLog: Assertion failed: 0 @ mmu_walk_unmap.c:72
2023-12-11T22:19:48.189433+01:00 desktop kernel: [   61.399227] NVRM nvAssertFailedNoLog: Assertion failed: pKernelBus->pReadToFlush != NULL || pKernelBus->virtualBar2[GPU_GFID_PF].pCpuMapping != NULL @ kern_bus_gv100.c:369
2023-12-11T22:19:48.237892+01:00 desktop systemd[1]: Starting systemd-suspend.service - System Suspend...
2023-12-11T22:19:48.258614+01:00 desktop systemd-sleep[3748]: Entering sleep state 'suspend'...
2023-12-11T22:19:48.261387+01:00 desktop kernel: [   61.469264] PM: suspend entry (deep)
2023-12-11T22:19:48.265396+01:00 desktop kernel: [   61.474105] Filesystems sync: 0.004 seconds
2023-12-11T22:20:08.416056+01:00 desktop kernel: [   61.605242] Freezing user space processes
2023-12-11T22:20:08.416199+01:00 desktop kernel: [   65.404670] NVRM dmaAllocMapping_GM107: can't alloc VA space for mapping.
2023-12-11T22:20:08.416200+01:00 desktop kernel: [   65.404694] NVRM kbusMapFbAperture_GM107: Failed: [GPU0] Could not map pAperOffset: 0x0
2023-12-11T22:20:08.416201+01:00 desktop kernel: [   65.404741] [drm:__nv_drm_gem_nvkms_map [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000b00] Failed to map NvKmsKapiMemory 0x00000000e4d10497
2023-12-11T22:20:08.416202+01:00 desktop kernel: [   65.409765] Freezing user space processes completed (elapsed 3.800 seconds)
2023-12-11T22:20:08.416203+01:00 desktop kernel: [   65.409769] OOM killer disabled.
2023-12-11T22:20:08.416203+01:00 desktop kernel: [   65.409770] Freezing remaining freezable tasks
2023-12-11T22:20:08.416204+01:00 desktop kernel: [   65.411143] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
2023-12-11T22:20:08.416206+01:00 desktop kernel: [   65.411164] printk: Suspending console(s) (use no_console_suspend to debug)
2023-12-11T22:20:08.416206+01:00 desktop kernel: [   65.444789] sd 6:0:0:0: [sda] Synchronizing SCSI cache
2023-12-11T22:20:08.416207+01:00 desktop kernel: [   65.445527] ata7.00: Entering standby power mode
2023-12-11T22:20:08.416208+01:00 desktop kernel: [   65.567492] r8169 0000:05:00.0 enp5s0: Link is Down
2023-12-11T22:20:08.416209+01:00 desktop kernel: [   66.149952] ACPI: EC: interrupt blocked
2023-12-11T22:20:08.416209+01:00 desktop kernel: [   66.200939] ACPI: PM: Preparing to enter system sleep state S3
2023-12-11T22:20:08.416210+01:00 desktop kernel: [   66.506359] ACPI: EC: event blocked
2023-12-11T22:20:08.416210+01:00 desktop kernel: [   66.506362] ACPI: EC: EC stopped
2023-12-11T22:20:08.416210+01:00 desktop kernel: [   66.506363] ACPI: PM: Saving platform NVS memory
2023-12-11T22:20:08.416211+01:00 desktop kernel: [   66.506440] Disabling non-boot CPUs ...
2023-12-11T22:20:08.416211+01:00 desktop kernel: [   66.508579] smpboot: CPU 1 is now offline
2023-12-11T22:20:08.416212+01:00 desktop kernel: [   66.511736] smpboot: CPU 2 is now offline
2023-12-11T22:20:08.416212+01:00 desktop kernel: [   66.514755] smpboot: CPU 3 is now offline
2023-12-11T22:20:08.416212+01:00 desktop kernel: [   66.517517] smpboot: CPU 4 is now offline
2023-12-11T22:20:08.416213+01:00 desktop kernel: [   66.520266] smpboot: CPU 5 is now offline
2023-12-11T22:20:08.416213+01:00 desktop kernel: [   66.523076] smpboot: CPU 6 is now offline
2023-12-11T22:20:08.416213+01:00 desktop kernel: [   66.525876] smpboot: CPU 7 is now offline
2023-12-11T22:20:08.416214+01:00 desktop kernel: [   66.528558] smpboot: CPU 8 is now offline
2023-12-11T22:20:08.416214+01:00 desktop kernel: [   66.531258] smpboot: CPU 9 is now offline
2023-12-11T22:20:08.416215+01:00 desktop kernel: [   66.533863] smpboot: CPU 10 is now offline
2023-12-11T22:20:08.416215+01:00 desktop kernel: [   66.536407] smpboot: CPU 11 is now offline
2023-12-11T22:20:08.416215+01:00 desktop kernel: [   66.538904] smpboot: CPU 12 is now offline
2023-12-11T22:20:08.416216+01:00 desktop kernel: [   66.541432] smpboot: CPU 13 is now offline
2023-12-11T22:20:08.416216+01:00 desktop kernel: [   66.543805] smpboot: CPU 14 is now offline
2023-12-11T22:20:08.416217+01:00 desktop kernel: [   66.546224] smpboot: CPU 15 is now offline
2023-12-11T22:20:08.416217+01:00 desktop kernel: [   66.548722] smpboot: CPU 16 is now offline
2023-12-11T22:20:08.416217+01:00 desktop kernel: [   66.549533] Spectre V2 : Update user space SMT mitigation: STIBP off
2023-12-11T22:20:08.416218+01:00 desktop kernel: [   66.551197] smpboot: CPU 17 is now offline
2023-12-11T22:20:08.416218+01:00 desktop kernel: [   66.553753] smpboot: CPU 18 is now offline
2023-12-11T22:20:08.416218+01:00 desktop kernel: [   66.556256] smpboot: CPU 19 is now offline
2023-12-11T22:20:08.416218+01:00 desktop kernel: [   66.558688] smpboot: CPU 20 is now offline
2023-12-11T22:20:08.416219+01:00 desktop kernel: [   66.561138] smpboot: CPU 21 is now offline
2023-12-11T22:20:08.416219+01:00 desktop kernel: [   66.563603] smpboot: CPU 22 is now offline
2023-12-11T22:20:08.416220+01:00 desktop kernel: [   66.565945] smpboot: CPU 23 is now offline
2023-12-11T22:20:08.416220+01:00 desktop kernel: [   66.568356] smpboot: CPU 24 is now offline
2023-12-11T22:20:08.416220+01:00 desktop kernel: [   66.570877] smpboot: CPU 25 is now offline
2023-12-11T22:20:08.416221+01:00 desktop kernel: [   66.573064] smpboot: CPU 26 is now offline
2023-12-11T22:20:08.416221+01:00 desktop kernel: [   66.575330] smpboot: CPU 27 is now offline
2023-12-11T22:20:08.416221+01:00 desktop kernel: [   66.577468] smpboot: CPU 28 is now offline
2023-12-11T22:20:08.416222+01:00 desktop kernel: [   66.579690] smpboot: CPU 29 is now offline
2023-12-11T22:20:08.416222+01:00 desktop kernel: [   66.581891] smpboot: CPU 30 is now offline
2023-12-11T22:20:08.416222+01:00 desktop kernel: [   66.584354] smpboot: CPU 31 is now offline
2023-12-11T22:20:08.416223+01:00 desktop kernel: [   66.585280] ACPI: PM: Low-level resume complete
2023-12-11T22:20:08.416223+01:00 desktop kernel: [   66.585297] ACPI: EC: EC started
2023-12-11T22:20:08.416223+01:00 desktop kernel: [   66.585298] ACPI: PM: Restoring platform NVS memory
2023-12-11T22:20:08.416224+01:00 desktop kernel: [   66.585435] LVT offset 0 assigned for vector 0x400
2023-12-11T22:20:08.416224+01:00 desktop kernel: [   66.585933] Enabling non-boot CPUs ...
2023-12-11T22:20:08.416224+01:00 desktop kernel: [   66.585961] smpboot: Booting Node 0 Processor 1 APIC 0x2
2023-12-11T22:20:08.416225+01:00 desktop kernel: [   66.588853] ACPI: \_PR_.C002: Found 2 idle states
2023-12-11T22:20:08.416225+01:00 desktop kernel: [   66.618204] CPU1 is up
2023-12-11T22:20:08.416225+01:00 desktop kernel: [   66.618222] smpboot: Booting Node 0 Processor 2 APIC 0x4
2023-12-11T22:20:08.416226+01:00 desktop kernel: [   66.620679] ACPI: \_PR_.C004: Found 2 idle states
2023-12-11T22:20:08.416226+01:00 desktop kernel: [   66.623841] CPU2 is up
2023-12-11T22:20:08.416226+01:00 desktop kernel: [   66.623867] smpboot: Booting Node 0 Processor 3 APIC 0x6
2023-12-11T22:20:08.416227+01:00 desktop kernel: [   66.626342] ACPI: \_PR_.C006: Found 2 idle states
2023-12-11T22:20:08.416227+01:00 desktop kernel: [   66.627827] CPU3 is up
2023-12-11T22:20:08.416227+01:00 desktop kernel: [   66.627850] smpboot: Booting Node 0 Processor 4 APIC 0x8
2023-12-11T22:20:08.416228+01:00 desktop kernel: [   66.630367] ACPI: \_PR_.C008: Found 2 idle states
2023-12-11T22:20:08.416228+01:00 desktop kernel: [   66.631828] CPU4 is up
2023-12-11T22:20:08.416228+01:00 desktop kernel: [   66.631850] smpboot: Booting Node 0 Processor 5 APIC 0xa
2023-12-11T22:20:08.416229+01:00 desktop kernel: [   66.634364] ACPI: \_PR_.C00A: Found 2 idle states
2023-12-11T22:20:08.416229+01:00 desktop kernel: [   66.635853] CPU5 is up
2023-12-11T22:20:08.416229+01:00 desktop kernel: [   66.635875] smpboot: Booting Node 0 Processor 6 APIC 0xc
2023-12-11T22:20:08.416230+01:00 desktop kernel: [   66.638402] ACPI: \_PR_.C00C: Found 2 idle states
2023-12-11T22:20:08.416230+01:00 desktop kernel: [   66.639885] CPU6 is up
2023-12-11T22:20:08.416230+01:00 desktop kernel: [   66.639907] smpboot: Booting Node 0 Processor 7 APIC 0xe
2023-12-11T22:20:08.416231+01:00 desktop kernel: [   66.642433] ACPI: \_PR_.C00E: Found 2 idle states
2023-12-11T22:20:08.416231+01:00 desktop kernel: [   66.644281] CPU7 is up
2023-12-11T22:20:08.416231+01:00 desktop kernel: [   66.644302] smpboot: Booting Node 0 Processor 8 APIC 0x10
2023-12-11T22:20:08.416231+01:00 desktop kernel: [   66.647117] ACPI: \_PR_.C010: Found 2 idle states
2023-12-11T22:20:08.416232+01:00 desktop kernel: [   66.649379] CPU8 is up
2023-12-11T22:20:08.416232+01:00 desktop kernel: [   66.649402] smpboot: Booting Node 0 Processor 9 APIC 0x12
2023-12-11T22:20:08.416232+01:00 desktop kernel: [   66.651950] ACPI: \_PR_.C012: Found 2 idle states
2023-12-11T22:20:08.416233+01:00 desktop kernel: [   66.655877] CPU9 is up
2023-12-11T22:20:08.416233+01:00 desktop kernel: [   66.655899] smpboot: Booting Node 0 Processor 10 APIC 0x14
2023-12-11T22:20:08.416234+01:00 desktop kernel: [   66.658441] ACPI: \_PR_.C014: Found 2 idle states
2023-12-11T22:20:08.416234+01:00 desktop kernel: [   66.659996] CPU10 is up
2023-12-11T22:20:08.416234+01:00 desktop kernel: [   66.660018] smpboot: Booting Node 0 Processor 11 APIC 0x16
2023-12-11T22:20:08.416234+01:00 desktop kernel: [   66.662565] ACPI: \_PR_.C016: Found 2 idle states
2023-12-11T22:20:08.416235+01:00 desktop kernel: [   66.664408] CPU11 is up
2023-12-11T22:20:08.416235+01:00 desktop kernel: [   66.664430] smpboot: Booting Node 0 Processor 12 APIC 0x18
2023-12-11T22:20:08.416235+01:00 desktop kernel: [   66.666974] ACPI: \_PR_.C018: Found 2 idle states
2023-12-11T22:20:08.416236+01:00 desktop kernel: [   66.671970] CPU12 is up
2023-12-11T22:20:08.416236+01:00 desktop kernel: [   66.671992] smpboot: Booting Node 0 Processor 13 APIC 0x1a
2023-12-11T22:20:08.416236+01:00 desktop kernel: [   66.674541] ACPI: \_PR_.C01A: Found 2 idle states
2023-12-11T22:20:08.416236+01:00 desktop kernel: [   66.676154] CPU13 is up
2023-12-11T22:20:08.416237+01:00 desktop kernel: [   66.676175] smpboot: Booting Node 0 Processor 14 APIC 0x1c
2023-12-11T22:20:08.416237+01:00 desktop kernel: [   66.678725] ACPI: \_PR_.C01C: Found 2 idle states
2023-12-11T22:20:08.416237+01:00 desktop kernel: [   66.680365] CPU14 is up
2023-12-11T22:20:08.416238+01:00 desktop kernel: [   66.680385] smpboot: Booting Node 0 Processor 15 APIC 0x1e
2023-12-11T22:20:08.416238+01:00 desktop kernel: [   66.682952] ACPI: \_PR_.C01E: Found 2 idle states
2023-12-11T22:20:08.416238+01:00 desktop kernel: [   66.684670] CPU15 is up
2023-12-11T22:20:08.416239+01:00 desktop kernel: [   66.684691] smpboot: Booting Node 0 Processor 16 APIC 0x1
2023-12-11T22:20:08.416239+01:00 desktop kernel: [   66.687242] ACPI: \_PR_.C001: Found 2 idle states
2023-12-11T22:20:08.416240+01:00 desktop kernel: [   66.692127] Spectre V2 : Update user space SMT mitigation: STIBP always-on
2023-12-11T22:20:08.416240+01:00 desktop kernel: [   66.692146] CPU16 is up
2023-12-11T22:20:08.416240+01:00 desktop kernel: [   66.692169] smpboot: Booting Node 0 Processor 17 APIC 0x3
2023-12-11T22:20:08.416241+01:00 desktop kernel: [   66.694737] ACPI: \_PR_.C003: Found 2 idle states
2023-12-11T22:20:08.416241+01:00 desktop kernel: [   66.696442] CPU17 is up
2023-12-11T22:20:08.416241+01:00 desktop kernel: [   66.696464] smpboot: Booting Node 0 Processor 18 APIC 0x5
2023-12-11T22:20:08.416241+01:00 desktop kernel: [   66.699034] ACPI: \_PR_.C005: Found 2 idle states
2023-12-11T22:20:08.416242+01:00 desktop kernel: [   66.701051] CPU18 is up
2023-12-11T22:20:08.416242+01:00 desktop kernel: [   66.701074] smpboot: Booting Node 0 Processor 19 APIC 0x7
2023-12-11T22:20:08.416243+01:00 desktop kernel: [   66.703656] ACPI: \_PR_.C007: Found 2 idle states
2023-12-11T22:20:08.416243+01:00 desktop kernel: [   66.708152] CPU19 is up
2023-12-11T22:20:08.416243+01:00 desktop kernel: [   66.708174] smpboot: Booting Node 0 Processor 20 APIC 0x9
2023-12-11T22:20:08.416244+01:00 desktop kernel: [   66.710712] ACPI: \_PR_.C009: Found 2 idle states
2023-12-11T22:20:08.416244+01:00 desktop kernel: [   66.712519] CPU20 is up
2023-12-11T22:20:08.416244+01:00 desktop kernel: [   66.712547] smpboot: Booting Node 0 Processor 21 APIC 0xb
2023-12-11T22:20:08.416244+01:00 desktop kernel: [   66.715129] ACPI: \_PR_.C00B: Found 2 idle states
2023-12-11T22:20:08.416245+01:00 desktop kernel: [   66.720213] CPU21 is up
2023-12-11T22:20:08.416245+01:00 desktop kernel: [   66.720234] smpboot: Booting Node 0 Processor 22 APIC 0xd
2023-12-11T22:20:08.416245+01:00 desktop kernel: [   66.722774] ACPI: \_PR_.C00D: Found 2 idle states
2023-12-11T22:20:08.416246+01:00 desktop kernel: [   66.724554] CPU22 is up
2023-12-11T22:20:08.416246+01:00 desktop kernel: [   66.724577] smpboot: Booting Node 0 Processor 23 APIC 0xf
2023-12-11T22:20:08.416246+01:00 desktop kernel: [   66.727155] ACPI: \_PR_.C00F: Found 2 idle states
2023-12-11T22:20:08.416247+01:00 desktop kernel: [   66.732301] CPU23 is up
2023-12-11T22:20:08.416247+01:00 desktop kernel: [   66.732321] smpboot: Booting Node 0 Processor 24 APIC 0x11
2023-12-11T22:20:08.416247+01:00 desktop kernel: [   66.734908] ACPI: \_PR_.C011: Found 2 idle states
2023-12-11T22:20:08.416248+01:00 desktop kernel: [   66.736814] CPU24 is up
2023-12-11T22:20:08.416248+01:00 desktop kernel: [   66.736836] smpboot: Booting Node 0 Processor 25 APIC 0x13
2023-12-11T22:20:08.416248+01:00 desktop kernel: [   66.739425] ACPI: \_PR_.C013: Found 2 idle states
2023-12-11T22:20:08.416248+01:00 desktop kernel: [   66.744388] CPU25 is up
2023-12-11T22:20:08.416249+01:00 desktop kernel: [   66.744409] smpboot: Booting Node 0 Processor 26 APIC 0x15
2023-12-11T22:20:08.416249+01:00 desktop kernel: [   66.747012] ACPI: \_PR_.C015: Found 2 idle states
2023-12-11T22:20:08.416249+01:00 desktop kernel: [   66.749170] CPU26 is up
2023-12-11T22:20:08.416250+01:00 desktop kernel: [   66.749192] smpboot: Booting Node 0 Processor 27 APIC 0x17
2023-12-11T22:20:08.416250+01:00 desktop kernel: [   66.751769] ACPI: \_PR_.C017: Found 2 idle states
2023-12-11T22:20:08.416250+01:00 desktop kernel: [   66.756443] CPU27 is up
2023-12-11T22:20:08.416250+01:00 desktop kernel: [   66.756464] smpboot: Booting Node 0 Processor 28 APIC 0x19
2023-12-11T22:20:08.416251+01:00 desktop kernel: [   66.759049] ACPI: \_PR_.C019: Found 2 idle states
2023-12-11T22:20:08.416251+01:00 desktop kernel: [   66.761769] CPU28 is up
2023-12-11T22:20:08.416251+01:00 desktop kernel: [   66.761791] smpboot: Booting Node 0 Processor 29 APIC 0x1b
2023-12-11T22:20:08.416252+01:00 desktop kernel: [   66.764386] ACPI: \_PR_.C01B: Found 2 idle states
2023-12-11T22:20:08.416252+01:00 desktop kernel: [   66.768569] CPU29 is up
2023-12-11T22:20:08.416252+01:00 desktop kernel: [   66.768591] smpboot: Booting Node 0 Processor 30 APIC 0x1d
2023-12-11T22:20:08.416252+01:00 desktop kernel: [   66.771201] ACPI: \_PR_.C01D: Found 2 idle states
2023-12-11T22:20:08.416253+01:00 desktop kernel: [   66.776520] CPU30 is up
2023-12-11T22:20:08.416253+01:00 desktop kernel: [   66.776542] smpboot: Booting Node 0 Processor 31 APIC 0x1f
2023-12-11T22:20:08.416253+01:00 desktop kernel: [   66.779146] ACPI: \_PR_.C01F: Found 2 idle states
2023-12-11T22:20:08.416254+01:00 desktop kernel: [   66.784570] CPU31 is up
2023-12-11T22:20:08.416254+01:00 desktop kernel: [   66.787932] ACPI: PM: Waking up from system sleep state S3
2023-12-11T22:20:08.416254+01:00 desktop kernel: [   66.788943] ACPI: EC: interrupt unblocked
2023-12-11T22:20:08.416255+01:00 desktop kernel: [   66.793432] ACPI: EC: event unblocked
2023-12-11T22:20:08.416255+01:00 desktop kernel: [   66.797304] nvme nvme1: Shutdown timeout set to 8 seconds
2023-12-11T22:20:08.416255+01:00 desktop kernel: [   66.797587] nvme nvme0: Shutdown timeout set to 8 seconds
2023-12-11T22:20:08.416255+01:00 desktop kernel: [   66.826492] nvme nvme0: 32/0/0 default/read/poll queues
2023-12-11T22:20:08.416260+01:00 desktop kernel: [   66.827745] nvme nvme1: 32/0/0 default/read/poll queues
2023-12-11T22:20:08.416260+01:00 desktop kernel: [   66.990937] r8169 0000:05:00.0 enp5s0: Link is Down
2023-12-11T22:20:08.416261+01:00 desktop kernel: [   67.001571] OOM killer enabled.
2023-12-11T22:20:08.420803+01:00 desktop kernel: [   67.001573] Restarting tasks ... done.
2023-12-11T22:20:08.420809+01:00 desktop kernel: [   67.002932] random: crng reseeded on system resumption
2023-12-11T22:20:08.420809+01:00 desktop kernel: [   67.002936] thermal thermal_zone0: failed to read out thermal zone (-61)
2023-12-11T22:20:08.421511+01:00 desktop systemd-resolved[1419]: Clock change detected. Flushing caches.
2023-12-11T22:20:08.421686+01:00 desktop gnome-shell[2274]: src/nv_gbm.c:127: GBM-DRV error (nv_gbm_bo_map): DRM_IOCTL_NVIDIA_GEM_MAP_OFFSET failed (ret=-1)
2023-12-11T22:20:08.421803+01:00 desktop gnome-shell[2274]: Realizing HW cursor failed: Failed write to gbm_bo: Onvoldoende geheugen beschikbaar
2023-12-11T22:20:08.421859+01:00 desktop gnome-shell[2274]: Failed to set hardware cursor (Failed write to gbm_bo: Onvoldoende geheugen beschikbaar), using OpenGL from now on
2023-12-11T22:20:08.436800+01:00 desktop kernel: [   67.022716] PM: suspend exit
2023-12-11T22:20:08.437038+01:00 desktop systemd-sleep[3748]: System returned from sleep state.
2023-12-11T22:20:08.437232+01:00 desktop bluetoothd[1461]: Controller resume with wake event 0x0
2023-12-11T22:20:08.528807+01:00 desktop kernel: [   67.113122] ata1: SATA link down (SStatus 0 SControl 300)
2023-12-11T22:20:08.693136+01:00 desktop kernel: [   67.275162] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
2023-12-11T22:20:08.693140+01:00 desktop kernel: [   67.275169] ata7.00: Entering active power mode
2023-12-11T22:20:08.944986+01:00 desktop kernel: [   67.529695] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.944990+01:00 desktop kernel: [   67.530492] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.944991+01:00 desktop kernel: [   67.530999] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.944991+01:00 desktop kernel: [   67.531515] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948810+01:00 desktop kernel: [   67.532020] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948814+01:00 desktop kernel: [   67.532965] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948816+01:00 desktop kernel: [   67.533472] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948817+01:00 desktop kernel: [   67.533978] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948817+01:00 desktop kernel: [   67.534481] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.948817+01:00 desktop kernel: [   67.534985] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952812+01:00 desktop kernel: [   67.535487] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952816+01:00 desktop kernel: [   67.536275] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952817+01:00 desktop kernel: [   67.536788] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952818+01:00 desktop kernel: [   67.537301] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952818+01:00 desktop kernel: [   67.537805] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952818+01:00 desktop kernel: [   67.538315] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952818+01:00 desktop kernel: [   67.538826] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.952819+01:00 desktop kernel: [   67.539345] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956812+01:00 desktop kernel: [   67.540132] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956816+01:00 desktop kernel: [   67.540636] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956817+01:00 desktop kernel: [   67.541144] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956817+01:00 desktop kernel: [   67.541650] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956818+01:00 desktop kernel: [   67.542160] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956818+01:00 desktop kernel: [   67.542663] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.956818+01:00 desktop kernel: [   67.543178] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960812+01:00 desktop kernel: [   67.543680] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960816+01:00 desktop kernel: [   67.544465] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960817+01:00 desktop kernel: [   67.544986] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960817+01:00 desktop kernel: [   67.545507] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960817+01:00 desktop kernel: [   67.546022] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960818+01:00 desktop kernel: [   67.546548] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.960818+01:00 desktop kernel: [   67.547063] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964812+01:00 desktop kernel: [   67.547584] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964817+01:00 desktop kernel: [   67.548101] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964817+01:00 desktop kernel: [   67.548620] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964818+01:00 desktop kernel: [   67.549135] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964818+01:00 desktop kernel: [   67.549654] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964818+01:00 desktop kernel: [   67.550167] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964819+01:00 desktop kernel: [   67.550686] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.964819+01:00 desktop kernel: [   67.551201] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968811+01:00 desktop kernel: [   67.551719] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968815+01:00 desktop kernel: [   67.552240] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968816+01:00 desktop kernel: [   67.552754] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968816+01:00 desktop kernel: [   67.553270] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968816+01:00 desktop kernel: [   67.553792] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968817+01:00 desktop kernel: [   67.554311] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968817+01:00 desktop kernel: [   67.554826] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.968817+01:00 desktop kernel: [   67.555351] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972809+01:00 desktop kernel: [   67.555873] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972814+01:00 desktop kernel: [   67.556388] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972815+01:00 desktop kernel: [   67.556909] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972815+01:00 desktop kernel: [   67.557424] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972815+01:00 desktop kernel: [   67.557937] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972815+01:00 desktop kernel: [   67.558456] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.972816+01:00 desktop kernel: [   67.558968] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.976807+01:00 desktop kernel: [   67.559480] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.976811+01:00 desktop kernel: [   67.559998] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.976812+01:00 desktop kernel: [   67.560516] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.976812+01:00 desktop kernel: [   67.561670] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.976813+01:00 desktop kernel: [   67.562793] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.981013+01:00 desktop kernel: [   67.563894] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.981017+01:00 desktop kernel: [   67.564994] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.981018+01:00 desktop kernel: [   67.566091] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.981018+01:00 desktop kernel: [   67.567185] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.985200+01:00 desktop kernel: [   67.568277] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.985205+01:00 desktop kernel: [   67.569373] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.985205+01:00 desktop kernel: [   67.570450] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.985206+01:00 desktop kernel: [   67.571527] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.989313+01:00 desktop kernel: [   67.572593] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.989318+01:00 desktop kernel: [   67.573665] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.989318+01:00 desktop kernel: [   67.574726] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.989319+01:00 desktop kernel: [   67.575785] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
2023-12-11T22:20:08.993312+01:00 desktop kernel: [   67.576843] NVRM kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70f000 returned garbage 0x0
...this repeats a lot...

edit: On the proprietary drivers suspend works as expected

veldenb avatar Dec 11 '23 21:12 veldenb

I also have the same issue on with nixos in the unstable channel. Nvidia 545 and from all kernel versions 6.x so far. I am also using the proprietary drivers

relief-melone avatar Dec 31 '23 22:12 relief-melone

Same here (same behaviour of veldenb's message), PCS laptop with RTX 3070 Gentoo, 6.7 kernel, 545.29.06 open driver

ElDavoo avatar Jan 14 '24 07:01 ElDavoo

Same issue here. Suspend doesn't work.

Environment:

  • NVIDIA proprietary driver on Debian 12, with X11 (not Wayland).
  • NOT using nvidia-suspend.sh, nvidia-hibernate.sh, nvidia-resume.sh.
$ uname -a
Linux doom 6.1.0-17-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux

$ systemctl list-unit-files | grep nvidia
nvidia-persistenced.service                enabled         enabled

# From journalctl
Jan 29 19:31:08 doom kernel: NVRM: GPU 0000:54:00.0: PreserveVideoMemoryAllocations module parameter is set. System Power Management attempted without driver procfs sus
pend interface. Please refer to the 'Configuring Power Management Support' section in the driver README.
Jan 29 19:31:08 doom kernel: nvidia 0000:54:00.0: PM: pci_pm_suspend(): nv_pmops_suspend+0x0/0x20 [nvidia] returns -5
Jan 29 19:31:08 doom kernel: nvidia 0000:54:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -5
Jan 29 19:31:08 doom kernel: nvidia 0000:54:00.0: PM: failed to suspend async: error -5

NVIDIA driver information:

$ nvidia-smi
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08              Driver Version: 545.23.08    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3080 Ti     On  | 00000000:54:00.0 Off |                  N/A |
|  0%   45C    P8              19W / 350W |    665MiB / 12288MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

miguno avatar Jan 29 '24 18:01 miguno

Same problem with resuming from hibernation for me.

  • Debian 12.4 on X11 Linux 6.1.0-17-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux
  • NVIDIA driver on a old GT640: NVIDIA-SMI 470.223.02 Driver Version: 470.223.02 CUDA Version: 11.4

With the 6.1.0-13, it works perfectly.

jofr49 avatar Jan 30 '24 16:01 jofr49

We are actively working upon it, shall update once the fix is integrated into release drivers.

amrit1711 avatar Jan 31 '24 06:01 amrit1711

Hey @amrit1711, It's been quite a few months. Any updates on the fix? Considering the open kernel modules will soon be the default and recommended for newer GPUs, getting this solved will allow many users to finally make the switch. I've been stuck using the proprietary drivers due to this.

fxzzi avatar May 16 '24 08:05 fxzzi

Hey @Fxzzi , thanks for the vigilance. This is definitely on the radar and being actively worked on. Sorry, can't really give any concrete dates, but very much stay tuned :)

mtijanic avatar May 16 '24 16:05 mtijanic

Thanks for the reply @mtijanic! Take your time, I'm just happy to see this is being worked on :)

fxzzi avatar May 16 '24 16:05 fxzzi

We missed calling it out in the changelog explicitly (oops), but this should be fixed with 555.42.02. Please test. I'll leave this bug open while 555.xx is still in beta.

mtijanic avatar May 21 '24 14:05 mtijanic

On 550.78, I've been suspending with some degree of success but with disabled Preserving. Let's see when 555 hits my distro. Thank you for the update.

ElDavoo avatar May 21 '24 14:05 ElDavoo