BentoBox icon indicating copy to clipboard operation
BentoBox copied to clipboard
verified publisher icon BentoBoxWorld

Locale files growing without bound

Open SlimeDog opened this issue 4 years ago • 14 comments

Expected behavior

Static locale file rewrites.

Observed/Actual behavior

BentoBox locale files are rewritten on every enable (ie., server start). Unfortunately the updated snakeyaml updates cause the rewrites to insert duplicate blank lines with every invocation. (Related but not new issue: long lines are split, compared with the reference files contained in the JAR.) This has two negative consequences:

  • the files grow without bound
  • they no longer compare to the reference files

Steps/models to reproduce

>version
[11:37:15] [Server thread/INFO]: This server is running CraftBukkit version 3395-Spigot-f4ff00f-cc86ab1 (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT)
[11:37:15] [Server thread/INFO]: You are running the latest version
  • Start server
  • Review locale files

BentoBox version

> bbox version
[11:32:03 INFO]: Running PAPER 1.18.1.
[11:32:03 INFO]: BentoBox version: 1.20.0-SNAPSHOT-b2208
[11:32:03 INFO]: Database: JSON
[11:32:03 INFO]: Loaded Game Worlds:
[11:32:03 INFO]: world_skyblock (SkyBlock): Overworld, Nether, The End
[11:32:03 INFO]: Loaded Addons:
[11:32:03 INFO]: Biomes 1.14.1-SNAPSHOT-b214 (ENABLED)
[11:32:03 INFO]: BSkyBlock 1.15.2-SNAPSHOT-b717 (ENABLED)
[11:32:03 INFO]: Challenges 1.0.0-SNAPSHOT-b474 (ENABLED)
[11:32:03 INFO]: ExtraMobs 1.12 (ENABLED)
[11:32:03 INFO]: Level 2.9.0-SNAPSHOT-b494 (ENABLED)

Plugin list

Irrelevant, but available on request

Other

No response

SlimeDog avatar Jan 06 '22 19:01 SlimeDog

Thanks for the info. I'll try and get to this on the weekend.

tastybento avatar Jan 07 '22 01:01 tastybento

Hmm, I can seem to replicate with the latest version of Paper or Spigot:

>version
[18:43:28] [Server thread/INFO]: This server is running CraftBukkit version 3397-Spigot-f4ff00f-f3828bb (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT)
[18:43:28] [Server thread/INFO]: Checking version, please wait...
[18:43:29] [Thread-10/INFO]: You are running the latest version

The locale files look normal. No blank lines or extraneous growth. Maybe this was a bug that was fixed in snakeml - https://github.com/PaperMC/Paper/pull/7248

tastybento avatar Jan 08 '22 02:01 tastybento 

>version
[23:47:38] [Server thread/INFO]: This server is running CraftBukkit version 3397-Spigot-f4ff00f-f3828bb (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT)
[23:47:38] [Server thread/INFO]: You are running the latest version
>bbox version
[23:47:25] [Server thread/INFO]: Running SPIGOT 1.18.1.
[23:47:25] [Server thread/INFO]: BentoBox version: 1.20.0-SNAPSHOT-b2210
[23:47:25] [Server thread/INFO]: Database: JSON
[23:47:25] [Server thread/INFO]: Loaded Game Worlds:
[23:47:25] [Server thread/INFO]: world_skyblock (SkyBlock): Overworld, Nether, The End
[23:47:25] [Server thread/INFO]: Loaded Addons:
[23:47:25] [Server thread/INFO]: Biomes 1.14.1-SNAPSHOT-b214 (ENABLED)
[23:47:25] [Server thread/INFO]: BSkyBlock 1.15.2-SNAPSHOT-b717 (ENABLED)
[23:47:25] [Server thread/INFO]: Challenges 1.0.0-SNAPSHOT-b474 (ENABLED)
[23:47:25] [Server thread/INFO]: ExtraMobs 1.12 (ENABLED)
[23:47:25] [Server thread/INFO]: Level 2.9.0-SNAPSHOT-b494 (ENABLED)

Blank line is still inserted above enums: and catalog: in plugins/BentoBox/locales/BentoBox//en-US.yml (and all other locales files, I think, but that's the one I monitor most closely), on every Spigot restart.

SlimeDog avatar Jan 08 '22 07:01 SlimeDog

https://hub.spigotmc.org/jira/browse/SPIGOT-6894

As mentioned in that - if you do a global search and replace for |+ and change it to just | then all the lines should disappear. I'm not sure whether md5 will agree that this is a bug, but I think it is.

tastybento avatar Jan 08 '22 23:01 tastybento

Spigot 1.18.1-3397 BentoBox-1.20.0-b2211

locales/en-US.yml unpacked from the JAR still contains 12 lines that end in |+. Are those that remain OK?

If in my local locales file long messages on split lines are joined to match those in the unpacked file, they are re-split when rewritten. For example (I remove formatting codes in my files):

      see-console-for-status: Purge started. See console for status or use &b /[label] purge status.

is rewritten as

      see-console-for-status: Purge started. See console for status or use /[label]
        purge status.

I assume that's OK, since it has been ever so.

The tail of the file

successfully-loaded: |

  &6   ____             _        ____
  &6  |  _ \           | |      |  _ \             &7 by &a tastybento &7 and &a Poslovitch
  &6  | |_) | ___ _ __ | |_ ___ | |_) | _____  __  &7 2017 - 2022
  &6  |  _ < / _ \ '_ \| __/ _ \|  _ < / _ \ \/ /
  &6  | |_) |  __/ | | | || (_) | |_) | (_) >  <   &b v&e [version]
  &6  |____/ \___|_| |_|\__\___/|____/ \___/_/\_\  &8 Loaded in &e [time]&8 ms.

fails on start-up, due to a missing 2. It should be:

successfully-loaded: |2

The files locales/ro.yml and locales/id.yml also fail, reporting empty paths.

[16:14:53] [Server thread/ERROR]: [BentoBox] Error updating locale file 'locales/ro.yml': Cannot set to an empty path
[16:14:53] [Server thread/ERROR]: [BentoBox] java.lang.IllegalArgumentException: Cannot set to an empty path
        at org.apache.commons.lang.Validate.notEmpty(Validate.java:321)
        at org.bukkit.configuration.MemorySection.set(MemorySection.java:190)
        at org.bukkit.configuration.file.YamlConfiguration.fromNodeTree(YamlConfiguration.java:158)
        at org.bukkit.configuration.file.YamlConfiguration.loadFromString(YamlConfiguration.java:115)
        at org.bukkit.configuration.file.FileConfiguration.load(FileConfiguration.java:160)
        at world.bentobox.bentobox.api.addons.Addon.getYamlFromJar(Addon.java:316)
        at world.bentobox.bentobox.managers.LocalesManager.updateLocale(LocalesManager.java:171)
        at world.bentobox.bentobox.managers.LocalesManager.lambda$copyLocalesFromAddonJar$0(LocalesManager.java:159)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at world.bentobox.bentobox.managers.LocalesManager.copyLocalesFromAddonJar(LocalesManager.java:155)
        at world.bentobox.bentobox.managers.AddonsManager.initializeAddon(AddonsManager.java:201)
        at world.bentobox.bentobox.managers.AddonsManager.loadAddon(AddonsManager.java:195)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at world.bentobox.bentobox.managers.AddonsManager.loadAddons(AddonsManager.java:138)
        at world.bentobox.bentobox.BentoBox.onEnable(BentoBox.java:152)
        at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
        at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:342)
        at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:480)
        at org.bukkit.craftbukkit.v1_18_R1.CraftServer.enablePlugin(CraftServer.java:526)
        at org.bukkit.craftbukkit.v1_18_R1.CraftServer.enablePlugins(CraftServer.java:440)
        at net.minecraft.server.dedicated.DedicatedServer.e(DedicatedServer.java:232)
        at net.minecraft.server.MinecraftServer.w(MinecraftServer.java:994)
        at net.minecraft.server.MinecraftServer.lambda$0(MinecraftServer.java:304)
        at java.base/java.lang.Thread.run(Thread.java:833)

SlimeDog avatar Jan 09 '22 00:01 SlimeDog

Thanks. Those things need to be fixed. I'll look at it today. From the Spigot bug it has been referred to the Snake YML project because it's a bug on them.

tastybento avatar Jan 09 '22 19:01 tastybento

I fixed the en-US.yml file - weirdly I did the others but not the English one (I think I changed it on the server but not the git repo). Anyway, hopefully this error will be fixed in snakeyaml project soon.

@SlimeDog How did you induce the errors with locales/ro.yml and locales/id.yml? I don't see any errors on loading and YAML checking them indicate not errors, so I'm not sure where that is coming from.

tastybento avatar Jan 09 '22 22:01 tastybento

Thanks for the en-US update.

Recipe for the errors:

  • Run under a Java 17.0.1 environment, on CentOS Linux
  • Delete plugins/BentoBox/locales/BentoBox/id.yml plugins/BentoBox/locales/BentoBox/ro.yml
  • Start server: Spigot or Paper 1.18.1 up-to-date
  • Review errors in console/log

Note: Neither file contains a valid snakeyml header (neither does hu.yml, but the "header" comment is treated as a node comment, I guess).

Note 2: Standard YAML validators (for example ymllint.org) do not validate for snakeyaml compliance.

SlimeDog avatar Jan 09 '22 23:01 SlimeDog

I'm not sure why, but I am seeing no errors from those files.

> bbox v
[19:50:21 INFO]: Running PAPER 1.18.1.
[19:50:21 INFO]: BentoBox version: 1.20.0-SNAPSHOT-b2212
[19:50:21 INFO]: Database: JSON
[19:50:21 INFO]: Loaded Game Worlds:
[19:50:21 INFO]: oneblock_world (OneBlock): Overworld, Nether*
[19:50:21 INFO]: Loaded Addons:
[19:50:21 INFO]: AOneBlock 1.9.0 (ENABLED)
[19:50:21 INFO]: Bank 1.4.0-SNAPSHOT-LOCAL (DISABLED)
[19:50:21 INFO]: Biomes 1.14.0 (ENABLED)
[19:50:21 INFO]: Border 3.2.0 (ENABLED)
[19:50:21 INFO]: Greenhouses 1.7.0-SNAPSHOT-b312 (ENABLED)
[19:50:21 INFO]: Limits 1.19.1-SNAPSHOT-LOCAL (ENABLED)
[19:50:21 INFO]: TwerkingForTrees 1.4.3-SNAPSHOT-LOCAL (ENABLED)
[19:50:21 INFO]: Upgrades 0.3.0-SNAPSHOT-LOCAL (ENABLED)
[19:50:21 INFO]: Warps 1.11.2 (ENABLED)
> version
[19:52:25 INFO]: Checking version, please wait...
[19:52:25 INFO]: This server is running Paper version git-Paper-142 (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT) (Git: a736c1a)
You are running the latest version

For the id.yml it literally has nothing in it except the flag. Are you sure the errors are coming from the BentoBox files?

tastybento avatar Jan 10 '22 03:01 tastybento

Are you sure the errors are coming from the BentoBox files?

That's what the console/log shows, per the sample ^^^

SlimeDog avatar Jan 10 '22 04:01 SlimeDog

Looking a little closer, the stack traces are for addons. Perhaps the addon locales/*/ files -- id and ro -- are the failure points, not the locales/BentoBox/. Those files do not exist in locales/Biomes/.

SlimeDog avatar Jan 10 '22 05:01 SlimeDog

I am still seeing

[Server thread/ERROR]: [BentoBox] Could not update locale file 'locales/ro.yml' due to it being malformed: unacceptable code point '' (0x0) special characters are not allowed

on

> bbox version
[21:45:00 INFO]: Running PAPER 1.18.2.
[21:45:00 INFO]: BentoBox version: 1.20.1-SNAPSHOT-b2235
[21:45:00 INFO]: Database: JSON
[21:45:00 INFO]: Loaded Game Worlds:
[21:45:00 INFO]: world_skyblock (SkyBlock): Overworld, Nether, The End
[21:45:00 INFO]: Loaded Addons:
[21:45:00 INFO]: Biomes 2.1.0-SNAPSHOT-b222 (ENABLED)
[21:45:00 INFO]: BSkyBlock 1.15.2-SNAPSHOT-b718 (ENABLED)
[21:45:00 INFO]: Challenges 1.0.0-SNAPSHOT-b475 (ENABLED)
[21:45:00 INFO]: ExtraMobs 1.12 (ENABLED)
[21:45:00 INFO]: Level 2.9.1-SNAPSHOT-b498 (ENABLED)

and

>bbox version
[21:45:55] [Server thread/INFO]: Running SPIGOT 1.18.1.
[21:45:55] [Server thread/INFO]: BentoBox version: 1.20.1-SNAPSHOT-b2235
[21:45:55] [Server thread/INFO]: Database: JSON
[21:45:55] [Server thread/INFO]: Loaded Game Worlds:
[21:45:55] [Server thread/INFO]: world_skyblock (SkyBlock): Overworld, Nether, The End
[21:45:55] [Server thread/INFO]: Loaded Addons:
[21:45:55] [Server thread/INFO]: Biomes 2.1.0-SNAPSHOT-b222 (ENABLED)
[21:45:55] [Server thread/INFO]: BSkyBlock 1.15.2-SNAPSHOT-b718 (ENABLED)
[21:45:55] [Server thread/INFO]: Challenges 1.0.0-SNAPSHOT-b475 (ENABLED)
[21:45:55] [Server thread/INFO]: ExtraMobs 1.12 (ENABLED)
[21:45:55] [Server thread/INFO]: Level 2.9.1-SNAPSHOT-b498 (ENABLED)

./plugins/BentoBox/locales/Challenges/ro.yml is empty. ./plugins/BentoBox/locales/Level/ro.yml is empty.

In the JAR, those files contain

hexdump -c locales/ro.yml
0000000  \n  \0  \0  \0                                                
0000004

SlimeDog avatar Mar 14 '22 04:03 SlimeDog

Thanks. They need to be delete. The latest BentoBox should record in the log the Addon responsible when it reports the error. Did it?

tastybento avatar Mar 14 '22 14:03 tastybento

Here are the relevant log entries. I identified the two culprits above:

  • ./plugins/BentoBox/locales/Challenges/ro.yml
  • ./plugins/BentoBox/locales/Level/ro.yml
>bbox reload
[09:11:44] [Server thread/INFO]: [BentoBox] Loading Settings from config.yml...
[09:11:44] [Server thread/INFO]: BentoBox > Settings reloaded.
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling addons...
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling BSkyBlock...
[09:11:44] [Server thread/INFO]: [Pladdon] Disabling Pladdon v1.15.2-SNAPSHOT
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling Level...
[09:11:44] [Server thread/INFO]: [BentoBox] [Level] Stopping Level queue
[09:11:44] [Server thread/INFO]: [Pladdon] Disabling Pladdon v1.0
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling ExtraMobs...
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling Challenges...
[09:11:44] [Server thread/INFO]: [Pladdon] Disabling Pladdon v1.0
[09:11:44] [Server thread/INFO]: [BentoBox] Disabling Biomes...
[09:11:44] [Server thread/INFO]: [BentoBox] Biomes addon disabled.
[09:11:44] [Server thread/INFO]: [Pladdon] Disabling Pladdon v1.0
[09:11:44] [Server thread/INFO]: [BentoBox] Addons successfully disabled.
[09:11:44] [Server thread/INFO]: [BentoBox] Loading addons...
[09:11:44] [Server thread/ERROR]: [BentoBox] Could not update locale file 'locales/ro.yml' due to it being malformed: unacceptable code point '' (0x0) special characters are not allowed
in "'reader'", position 1
[09:11:44] [Server thread/INFO]: [BentoBox] Loaded 5 addons.
[09:11:44] [Server thread/INFO]: [BentoBox] Enabling game mode addons...
[09:11:44] [Server thread/INFO]: [BentoBox] Enabling BSkyBlock (1.15.2-SNAPSHOT-b718)...
[09:11:44] [Server thread/INFO]: Multiverse already knows about 'world_skyblock'!
[09:11:44] [Server thread/INFO]: Success! Property generator was set to BentoBox
[09:11:44] [Server thread/INFO]: Multiverse already knows about 'world_skyblock_nether'!
[09:11:44] [Server thread/INFO]: Success! Property generator was set to BentoBox
[09:11:44] [Server thread/INFO]: Multiverse already knows about 'world_skyblock_the_end'!
[09:11:44] [Server thread/INFO]: Success! Property generator was set to BentoBox
[09:11:44] [Server thread/INFO]: [BentoBox] Enabling other addons...
[09:11:44] [Server thread/INFO]: [BentoBox] Enabling Level (2.9.1-SNAPSHOT-b498)...
[09:11:45] [Server thread/INFO]: [BentoBox] [Level] Level hooking into BSkyBlock
[09:11:45] [Server thread/INFO]: [PlaceholderAPI] Successfully registered expansion: level [2.9.1-SNAPSHOT-b498]
[09:11:45] [Server thread/INFO]: [BentoBox] Enabling ExtraMobs (1.12)...
[09:11:45] [Server thread/INFO]: [BentoBox] Enabling Challenges (1.0.0-SNAPSHOT-b475)...
[09:11:45] [Server thread/INFO]: [BentoBox] [Challenges] Loading challenges...
[09:11:45] [Server thread/INFO]: [PlaceholderAPI] Successfully registered expansion: bskyblock [1.15.2-SNAPSHOT-b718]
[09:11:45] [Server thread/INFO]: [BentoBox] Enabling Biomes (2.1.0-SNAPSHOT-b222)...
[09:11:45] [Server thread/INFO]: [BentoBox] Loading biomes...
[09:11:46] [Server thread/INFO]: [BentoBox] Addons successfully enabled.
[09:11:46] [Server thread/INFO]: BentoBox > Addons reloaded.
[09:11:47] [Server thread/ERROR]: [BentoBox] Could not update locale file 'locales/ro.yml' due to it being malformed: unacceptable code point '' (0x0) special characters are not allowed
in "'reader'", position 1
[09:11:47] [Server thread/INFO]: BentoBox > Languages reloaded.
[09:11:47] [Server thread/INFO]: [BentoBox] [Challenges] Challenges Addon hooked into Level addon.
[09:11:47] [Server thread/INFO]: [BentoBox] [Challenges] Challenges Addon hooked into Economy.
[09:11:47] [Server thread/INFO]: [BentoBox] [Biomes] Biomes Addon hooked into Level addon.
[09:11:47] [Server thread/INFO]: [BentoBox] [Biomes] Biomes Addon hooked into Economy.
[09:11:47] [Server thread/INFO]: [BentoBox] Added world SkyBlock (HARD)
[09:11:47] [Craft Scheduler Thread - 504/INFO]: [BentoBox] [Level] Generating rankings
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded Blueprint Bundle 'harder_island' for BSkyBlock.
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded Blueprint Bundle 'default' for BSkyBlock.
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded Blueprint Bundle 'double_island' for BSkyBlock.
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded blueprint 'nether-island' for BSkyBlock
[09:11:47] [Craft Scheduler Thread - 504/INFO]: [BentoBox] [Level] Generated rankings for world_skyblock
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded blueprint 'end-island' for BSkyBlock
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded blueprint 'harder' for BSkyBlock
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded blueprint 'island' for BSkyBlock
[09:11:47] [Craft Scheduler Thread - 505/INFO]: [BentoBox] Loaded blueprint 'double' for BSkyBlock

SlimeDog avatar Mar 14 '22 16:03 SlimeDog

I haven't seen these issues for a while now.

SlimeDog avatar Oct 01 '22 23:10 SlimeDog