Persistence all around bug (seems like) after server update 1.27

[vincentroyc]

Updated all my server for 1.27 with latest mods : CF, Community Online Tools and SchanaModCompass only.

I was having problems overall persistence, like thirst and food levels, bullet stack quantities, item damage levels and more after server restarts. Server update had wiped also.

I'd get logs like these for a LOT if not all items types when restarting :

14:02:22 !!! Scripted variables corrupted upon "PUScopeOptic"
14:02:22 SCRIPT    (E): [EntityAI::OnStoreLoad] :: [WARNING] :: Scripted variables corrupted upon "PUScopeOptic".
Entity will not be loaded correctly.
Function: 'main'
Stack trace:
$CurrentDir:mpmissions/dayzOffline.chernarusplus/init.c:6
14:02:22 !!! Scripted variables corrupted upon "PUScopeOptic"

I went down tried to narrow down the problem.

Attempt #1

• Took one of my buggy server setup as is
• wiped all storage and profiles
• removed all mods
• tested it modless
• Everything seemed good after restart of server and in-game test

Attempt #2 (adding mods one by one)

• Took one of my buggy server setup as is
• wiped all storage and profiles
• Added CF only as mods
• tested it
• Began to have the same errors as mentioned when restarting the server
• And in-game tests after a restart I had every same problems like bullet stack going from a few to max stacks and all other persistence problems

Hope this helps

[bzed]

We had the same issue, but after wiping the servers so far I did not hear anything about it being back.

Bloody hands under gloves were also a common symptom.

[vincentroyc]

We had the same issue, but after wiping the servers so far I did not hear anything about it being back.

Bloody hands under gloves were also a common symptom.

How do you wipe your servers? Because I made multiple tests wiping the server with and without the mod and I always get the issue with the mod added (only this mod added in tests)

I wipe by deleting the entire storage and profiles directories.

[bzed]

How do you wipe your servers? Because I made multiple tests wiping the server with and without the mod and I always get the issue with the mod added (only this mod added in tests)

I wipe by deleting the entire storage and profiles directories.

Same. Stop server, remove storage_1 and all other directories (like ATM settings) that need to do away.

Maybe it's a bug that happens with a lot of traffic or a lot of players - my servers are not that crowded at the moment.

[bzed]

Actually I can confirm it - after some time the first players ran into the same issue. Also we have some random client crashes that are related to the loading of their state while logging in/

[dgbretro]

I am having the same issue. Server without CF, no problems. Servers with CF. Persistence bug. I have tried everything at this point, including a complete rebuild of the servers themselves. Adding CF before the server even starts. Def keeps happening

[bzed]

@lava76 any ideas on that? I would be happy to help debug.

[dgbretro]

Just completely wiped a server, removed and reinstalled DayZ server software. No mods aside from CF, Occurred immediately after the first restart. Only happens after a restart, you can log out and log in and no issues with inventory. Server restarts and the persistence issues happen.

[dgbretro]

Having the same issue on a windows server as well

[MSchmitt98]

Me too. My server opened up this week so it's basically all new with only a couple of mods. I even kick all players like 5 min before restart and lock the server in order to save all the data properly but it doesn't matter. The issue is always after the restart. No issues when not loading CF.

[bzed]

https://feedback.bistudio.com/T189290 - this could be related (or just be the same bug, not sure)

[lava76]

I think T189290 may not be related in this case. Try the latest CF-Test release (1.5.6 RC1) on the Steam Workshop. Wipe complete storage_x directory beforehand. If you still have issues, attach only the script.log to this ticket. Thanks.

[lava76]

Also, always make sure the server writes its complete storage at least once after it has fully booted up the first time after adding CF (should take around 30 seconds, you can check the timestamps of the files in the storage_x/data folder to be sure, when you see the time of all files change to the current time, the storage has been written at least once).

[bzed]

Ran with 5 people on one of my dev servers with the CF-Test mod installed and we didn't have any issues. Restarted 6 times in between, all fine. Thanks @lava76

[dgbretro]

It's not happening on 2/3 of my servers. Installed the CF-Test on the 1 affected. Wiped storage_x before hand. Still happening.

script_2025-03-10_16-28-04.log

[lava76]

In the script log I can clearly see that the storage was not wiped.

[dgbretro]

I must be missing something, delete storage_x from the missions folder and restart. Is there another step?

[lava76]

Are you sure you deleted the right storage?

[lava76]

Also, server can't be running while deleting storage. Stop it, delete folder, then start. Just in case that's not clear.

[dgbretro]

Actually the script I gave you is from when I noticed the bug (happens after first restart). So my test case scenario is this 1)Shut down 2)Delete Storage_x 3) Start up 4)Log into server, find ammo or use admin tools to get ammo or raw fruit or meat 5) log out 6)log in, check inventory Hang on, just wiped, will attach script after fresh wipe EDIT: Attached is the script log after wipe and 1st login

script_2025-03-10_17-05-31.log

[lava76]

That looks fine.

[dgbretro]

The issue doesnt happen until after a restart. I am not entirely convinced it's a serverside issue to be honest. Testing with another server admin we found that when setting up a brand new server (brand new AWS instance) ONLY mod is CF. I would get the bug, but he wouldnt. Tested on one of my servers, once again he did not experience the bug. I blew away DayZ client side on my end and reinstalled. And 2 of 3 servers I am not experiencing it. The only server I see it on now is Sakhal. EDIT: So if everything looks fine on your end, logically, that would suggest it is client side

[lava76]

Is your storage directory, especially the directory storage_x/communityframework, write protected by any chance?

[dgbretro]

Let me check that

[dgbretro]

Owner has read/write/execute Group has R/X

[lava76]

Which user does the DayZ server process run under? Does it have write access?

[lava76]

Specifically, do you see the file modstorageplayers.bin in there, and is it larger than 0 bytes after you connected for the 1st time and shut down the server?

[dgbretro]

The server is through a MSP, Given that the other server I have with them is not having the issue I would say the user that is running the process is the owner of the directory and has write access. The modstorageplayers.bin is 0 bytes after logging in for 1st time

[lava76]

Does it stay 0 bytes after shutting down the server? Because the file is only flushed after the server exits.

[bzed]

Does it stay 0 bytes after shutting down the server? Because the file is only flushed after the server exits.

What do you mean that it is only flushed when the server exits? If you write it only on exit, that is most likely a reason for a fail. Depending on the environment it is not necessarily given that the OS waits long enough for a process to finish. Also you might not have it under control that the server process waits long enough till a file is being written before it will exit. Last but not least it might make a difference if I send a shutdown command via rcon vs. sending the process a TERM signal. For example running in a container gives you 10s of time to shutdown the server by default.

[dgbretro]

It goes to 1kb after shutting down the server, and stays 1kb after starting back up.