Upgrade to Pydantic v2

[vberlier]

Beet is upgrading to Pydantic v2. I wanted to make sure that the changes on the beta branch don't break anything so I decided to try to build gm4 with it. And with this PR I'm successfully building everything:

$ uv run beet -p beet-release.yaml

The dev command seems to work as well.

Nearly all of the changes I had to make were related to Pydantic. Some things to note:

• I had to patch some overlay handling in the backwards compatibility plugin (crash when overlay_formats is None). This is most likely from recent changes on beet's main branch that weren't accounted for in gm4 yet.
• skin_cache.json got updated, not sure why?
• In Pydantic v2 nullability no longer implies that the field can be omitted. We need explicit None default values.
• I had to convert the validation of TemplateOptions, TransformOptions, and ContainerGuiOptions to a sort of dynamic discriminated union.
• Pydantic v2 doesn't include fields from derived classes when converting to a dict if you don't specify serialize_as_any=True.
• Pydantic's ValidationError changed and I couldn't easily convert the code responsible for aggregating and prefixing errors from NestedModelData validation. Now, if there are errors, only the first one is shown.

Feel free to polish this up, or just use the PR as a guide and completely re-do things your way.

[SpecialBuilder32]

Thanks VB! Like 2 years ago I took a stab at updating our admittedly complex RP build process to pydantic 2.... And everything broke very badly. I'll dig up my old branch and compare notes, hopefully get this working pretty soon.

There's complexity in our config broadcast system for item models, to prevent repeating the same lines in config for every item variant.

[vberlier]

Here's the manifest generated by the release build on my machine release\1.21.5\meta.json Do you think you could check if it's still the same?

[vberlier]

Alright, the manifests don't match. All modules hashes are different.

All pack.mcmeta have changes:

--- a/baseline/gm4_bat_grenades_1_21_5/pack.mcmeta
+++ b/baseline/gm4_bat_grenades_1_21_5/pack.mcmeta
@@ -1,6 +1,13 @@
 {
   "pack": {
-    "pack_format": 71,
+    "min_format": [
+      88,
+      0
+    ],
+    "max_format": [
+      88,
+      0
+    ],
     "description": [
       "Bat Grenades",
       "\n",
@@ -9,11 +16,12 @@
         "color": "#4AA0C7"
       }
     ],
+    "pack_format": 71,
     "supported_formats": {
       "min_inclusive": 71,
       "max_inclusive": 81
     }
   },
   "version": "1.6.0",
-  "commit_hash": "cb0fe4537"
+  "commit_hash": "14118d19"
 }

The added min_format and max_format are from beet's main branch.

All png files have changes. The png files are visually identical but the new python/pillow version probably changed the encoding/compression slightly.

All structure files have changes. The nbt data is the same, but again the files likely differ due to minor adjustments in python's compression libraries.

Some changes to gm4_resource_pack_1_21_5/assets/gm4_translations/lang/en_us.json. There's some minor reordering but it looks like a few translations were previously missing? e.g. block.gm4.custom_crafter

Some changes in most item models. It looks like it's fixing a potential issue where you currently get redundant entries? For example the previous version of gm4_resource_pack_1_21_5/assets/minecraft/items/player_head.json doesn't reference cool_empty, hot_empty and cool_metal anywhere.

--- a/baseline/gm4_resource_pack_1_21_5/assets/minecraft/items/player_head.json
+++ b/baseline/gm4_resource_pack_1_21_5/assets/minecraft/items/player_head.json
@@ -162,7 +162,7 @@
           "model": {
             "type": "minecraft:player_head"
           },
-          "base": "gm4_metallurgy:block/mould/hot_metal"
+          "base": "gm4_metallurgy:block/mould/cool_empty"
         }
       },
       {
@@ -172,7 +172,7 @@
           "model": {
             "type": "minecraft:player_head"
           },
-          "base": "gm4_metallurgy:block/mould/hot_metal"
+          "base": "gm4_metallurgy:block/mould/hot_empty"
         }
       },
       {
@@ -182,7 +182,7 @@
           "model": {
             "type": "minecraft:player_head"
           },
-          "base": "gm4_metallurgy:block/mould/hot_metal"
+          "base": "gm4_metallurgy:block/mould/cool_metal"
         }
       },
       {

That's all the changes I gathered from the release output.

[misode]

@vberlier What's the status of the beta beet branch? Am I correct in guessing that we need to wait for that before we attempt to merge this PR?

[SpecialBuilder32]

@misode this branch will also need some heavy work from me I suspect. The RP relies on pydantic features quite a bit and will need some validators rewritten in the pydanticv2 paradigm.

[vberlier]

@misode The beta branch includes all the commits from main, with the pydantic v2 migration, the update to python 3.14, plus a couple of related fixes. It's ready to use for any project updating to pydantic v2 and python 3.14. There are no major changes in beet itself, apart from the monorepo structure but that doesn't impact users. I'm not merging the beta branch right now to give people some time to deal with the pydantic v2 breakage.

But I'm probably only going to wait for a few more weeks. Projects that rely heavily on pydantic need to try switching to the beta branch as soon as possible so that they can be ready for pydantic v2 when it's released on main.

@SpecialBuilder32 In this PR I already rewrote your models and validators to make everything work with pydantic v2. As I noted earlier, there are a some minor changes in the compiled output, but those are either inconsequential or appear to actually fix a potential bug in the previous implementation. But you'll probably want to re-do some things your way, or at least double check that everything still works as expected and merge this PR as-is until you can take some time to revisit things and maybe do a more extensive revamp to leverage pydantic v2 more idiomatically.

[misode]

The dev command seems to work as well.

~~I hadn't tried this PR before merging in master, but it doesn't work for me. The Pygantic validators for TemplateOptions don't seem to work. I might need your help with this resource pack plugin @SpecialBuilder32~~

Update: I found a fix and the PR builds now.

[SpecialBuilder32]

Alright I've had some time to dig my teeth into pydantic v2 and these changes. Thanks @vberlier and @misode for doing all the work, everything seems to work perfectly.

The new validator code works for all the broadcast cases I could throw at it, and I did an output RP comparison between main and this PR and found no meaningful differences, which gives me confidence the config is resolving identically.

Is there any lingering issues you want me to look at, and are we waiting on any upstream beet changes before this can/needs to get merged?