AV when background saving

[ramakrishnanjayaraman]

Hi,

We are constantly facing these AV errors (event logs filled most of the times) ...happening since 3.2.100 and even with this fork, Might you please be able to help out ?

=== REDIS BUG REPORT START: Cut & paste starting from here === Redis version: 4.0.2.3 [2088] 28 Sep 09:49:34.120 # --- EXCEPTION_ACCESS_VIOLATION [2088] 28 Sep 09:49:34.120 # --- STACK TRACE redis-server.exe!((null):0)(0x00000000, 0x00000000, 0x00000001, 0x00000000) KERNELBASE.dll!UnhandledExceptionFilter((null):0)(0x00000000, 0x7FFFAE0B1DC0, 0x00000000, 0x0013FF20) ntdll.dll!memset((null):0)(0x0013EEB0, 0x0013E9C0, 0x1401707B0, 0x00000000) ntdll.dll!_C_specific_handler((null):0)(0x00000000, 0x0013E3F0, 0x0013EEB0, 0x0013EEB0) ntdll.dll!_chkstk((null):0)(0x00000001, 0x7FFFB0C90000, 0x00000000, 0x7FFFB0DC409C) ntdll.dll!RtlRaiseException((null):0)(0x002325E0, 0x00000000, 0x0013F240, 0x00000000) ntdll.dll!KiUserExceptionDispatcher((null):0)(0x02407040, 0x00000000, 0x00004065, 0x00000000) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x0013F240, 0x00000000, 0x00000000, 0x65400000) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x00270CB0, 0x00760000, 0x00B62760, 0x00000828) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x0000000A, 0x00760000, 0x00760000, 0x000000F8) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x00000000, 0x00237880, 0x00236750, 0x1401B4350) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x00237880, 0x00000000, 0x00000005, 0x00000006) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x140045D00, 0x00000000, 0x1401B8F80, 0x7FF5FFFFA000) redis-server.exe!KiUserExceptionDispatcher((null):0)(0x140045D18, 0x7FF5FFFFA000, 0x00000000, 0x00000000) KERNEL32.DLL!BaseThreadInitThunk((null):0)(0x7FFFAE5213B0, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart((null):0)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart((null):0)(0x00000000, 0x00000000, 0x00000000, 0x00000000) [2088] 28 Sep 09:49:34.120 # --- INFO OUTPUT [2876] 28 Sep 09:49:34.479 # fork operation failed [2876] 28 Sep 09:49:34.495 # Background saving terminated by signal 1

[israellot]

In what setup are you running? The fork operation failed. In the windows port, as there's not a kernel fork like Linux, it means a copy-on-write operation involving the pagefile on disk. So if your pagefile can't handle the Redis dataset, it might throw an exception back.

[ramakrishnanjayaraman]

In what setup are you running? The fork operation failed. In the windows port, as there's not a kernel fork like Linux, it means a copy-on-write operation involving the pagefile on disk. So if your pagefile can't handle the Redis dataset, it might throw an exception back.

Thank you for the quick response! Windows 8 and Windows 10 both demonstrate the same behavior. How do i check if it's due to the pagefile ? and how / what should i do to change it so it'll not fail ?

[ramakrishnanjayaraman]

Also note that my dataset size is hardly 50mb. I assume page file will really not be an issue at this size ? Could it be due to the save configuration i have ?

180 1 60 10

[tporadowski]

@ramakrishnanjayaraman is this reproducible or happens randomly? Since your data set is not big - is there a chance to share it for debugging purposes?

[ramakrishnanjayaraman]

Hi @tporadowski , yes, i can surely share it. And it happens pretty regularly. If i increase the time window to simply something like 180 1 and not do the 60 10 then it seems to work a bit more reliably.

Let me know how best to share with you for debugging purposes ?

[tporadowski]

Easiest would be to email it to me (ZIP your data and config) to [email protected] or use some OneDrive, DropBox or Box service and send me the share link. Please also share some details on what operations are you performing on that data as it might be important (can also be in the email).

[israellot]

A small dataset makes less likely the pagefile theory. Please share the whole config file used. Are you using AOF or RDB or both?

[ramakrishnanjayaraman]

Easiest would be to email it to me (ZIP your data and config) to [email protected] or use some OneDrive, DropBox or Box service and send me the share link. Please also share some details on what operations are you performing on that data as it might be important (can also be in the email).

Hi @tporadowski , Send you an email with the log, database dump and configuration.

Let me know if i can provide further details.

Regards, Ram

[ramakrishnanjayaraman]

A small dataset makes less likely the pagefile theory. Please share the whole config file used. Are you using AOF or RDB or both?

Hi @israellot , Here's the config. WE only use RDB.

Regards, Ram

[virupaksh1984]

@tporadowski I'm also facing same issue. How do we set maxheap in this version of redis

[nautilux2]

Is there any new info?

[tporadowski]

@natilux2 unfortunately, no. I remember playing around with data received from @ramakrishnanjayaraman, but could not reproduce it. Are you also experiencing this on the latest v4.0.14.2 version?

[nautilux2]

@tporadowski Currently, I'm trying this v4.0.14.2 version and sometimes got that exception. I will send you dump file or error next time I got it

[tporadowski]

Great, will investigate it then again.