SkriptLang
Make the 5-min variable saving period configurable
Description
Variables in Skript are not getting auto saved to variables.csv file until the server is rebooted. The newly created variables or changes which although works correctly in the skript in-game, are not saved into the variables.csv file automatically. So, If the server crashes at some point, the variables are set back to the older values since the autosaving is not working. The new values are saved and shown properly in game but arent reflected in the variables.csv file. I have Skript version 2.3.6 running on Paper 1.13.2. This problem causes issues since upon a server crash due to some other reason, the variable are set to the values which existed in the previous reboot causing much data loss.
Steps to Reproduce
Save any variable(I used a list -> {eggcount::%player%} ) The variables are not saved to me in the CSV file until the server reboots. All new data or variables are lost if server crashes and gets reset to the values which were on the previous restart.
Expected Behavior
Variables should be saved to variables.csv at intervals/immediately when a change occurs.
Errors / Screenshots
No Errors in code. Variables aren't saved to CSV file until server reboot --->
Server Information
- Server version/platform: 1.13.2 Paper Spigot
- Skript version: Skript 2.3.6
My config.sk file. I havent changed anything here. I was able to reproduce the same issue on a diffrent freshly setup 1.13.2 server `
type: CSV
pattern: .*
file: ./plugins/Skript/variables.csv
backup interval: 2 hours
# PS: If you don't want some variables to be saved in any database (e.g. variables that contain an %entity% which usually despawn when the server is shut down)
# you can modify the last database's pattern to not match all variables, e.g. use '(?!x_).*' to match all variables that don't start with 'x_'.
# Be very cautious when doing this however as unsaved variables cannot be recovered after the server has been stopped.
# I recommend to use a single character to denote unsaved variables (similar to local variables' '_'), e.g. '-', in which case the last database's pattern should be '(?!-).*'.
`
I don't know if this is true and (if it is) if there's anything that can be done about it, so I'll defer to devs
I noticed the issue only when my server crashed a couple of times each time resetting all the variables to the state in which it was during the previous proper shutdown
Variables are not saved immediately; data loss may occur if the server crashes. They should be saved periodically, though. If this is not happening, we have a bug.
Variables are not saved immediately; data loss may occur if the server crashes. They should be saved periodically, though. If this is not happening, we have a bug.
They are definitely saved periodically, though, that isn't exactly efficient so there's a room for improvement in that section, or to be fair, the variable storage system itself.
As for this issue, there's not really much we can do when a crash happens, data lost is something that should be totally expected on these cases. If anything, I'd look at the causes of the crash since that isn't normal in a production server.
It would be awesome if you could somehow add a flag for a variable to be saved immediately on modify. We could use this for variables that are not changed too frequently. A new type of variable which is unlike the other(which are saved at intervals) for highly reliable storage by making sure it's saved to file when modified.
They are definitely saved periodically
Not correct, at least on most recent SK jar. @RishabhRaghunath has correctly reported this issue. And, it is a major issue. I'd call it a CRITICAL issue since its ramification is the complete loss of all SK game data (variables) since the last server restart. I've completely verified these findings. Variables.csv is not written to until the server is properly stopped, which is highly dangerous for user data. Please address with priority.
Variables are supposed to be saved every 5 minutes if 1000 or more changes to the variables have been made, and works as expected for me. But i would say these settings should be configurable.
https://github.com/SkriptLang/Skript/blob/master/src/main/java/ch/njol/skript/variables/FlatFileStorage.java#L196-L201
Variables are supposed to be saved every 5 minutes if 1000 or more changes to the variables have been made, and works as expected for me. But i would say these settings should be configurable.
https://github.com/SkriptLang/Skript/blob/master/src/main/java/ch/njol/skript/variables/FlatFileStorage.java#L196-L201
Like you said, should be entirely configurable so that servers can save more regularly for data preservation based on their own requirements.
Variables are supposed to be saved every 5 minutes if 1000 or more changes to the variables have been made, and works as expected for me. But i would say these settings should be configurable.
https://github.com/SkriptLang/Skript/blob/master/src/main/java/ch/njol/skript/variables/FlatFileStorage.java#L196-L201
@Blueyescat and @RishabhRaghunath , I confirm that the current method (in 2.4-beta5) saves variables to file variables.csv within a moving 5 minute window IF there have been at least 1000 variable writes since the last 5 minute window. I tested this several times using a simple loop to change a single variable's value 1000 times (via a command), then waited to see the file change.
EDIT: Short of that 1000 variable-writes threshold, variables.csv does not appear to be updated on disk until a safe server stop has occurred.
For smaller servers where 1000 variable changes or a server restart may not happen for a long while, this could be disastrous in the event of a server crash leading to the loss of hours, days or weeks of game data.
So yes, this must be given priority and made configurable to suit individual server needs.
Thanks.