I have a big pile of improvements to upstream

[camillol]

I upgraded a large private codebase to Pydantic 2, and in the process I made a bunch of improvements to bump-pydantic. See this branch, and the list of commits.

Here is a list of issues this branch addresses (possibly slightly incomplete, it's lightly edited from a personal log):

• [x] allow_mutation in Config should be converted to frozen https://github.com/pydantic/bump-pydantic/pull/161
• [x] json_encoders is allowed again, no need for comment https://github.com/pydantic/bump-pydantic/pull/162
• [x] doe not handle values param in validators
• [x] when replacing validators, classmethod can be duplicated https://github.com/pydantic/bump-pydantic/pull/160
• [x] stops everything on the first parse error instead of continuing
• [x] validator(always=True) should be converted to validate_default=True on Field
• [x] Add the ability to migrate Ormar models (for ormar 0.20.0)
• [x] Class definition scanner logic is wrong, e.g. if you have A(B) B(C) C(BaseModel) across three modules
• [x] test_class_def_visitor.py is commented out
• [x] should ignore .git
• [x] tests don’t run ClassDefVisitor
• [x] It thinks any nested Config class is a pydantic config. It should check that the parent is actually a model.
• [x] root_validator gets turned into a model_validator without arguments, which is invalid (needs mode).
• [x] we handled that for root_validator, now also need to handle root_validator()
• [x] smart_union is now the default. It should be automatically removed.
• [x] underscore_attrs_are_private is also now the default and should be removed.
• [x] Scanning for classes takes forever on a large repo, should use Pyre
• [x] In Pydantic 1, the type of fields could be inferred from the default value. In 2, they must have a type annotation.
  • [x] Done for simple types of literal.
  • [x] Added support for type inference.
  • [x] Reduce the FQN for things from the current module
• [x] BaseModel.json is deprecated, should use model_dump_json.
• [x] parse_obj also deprecated. use model_validate
• [x] construct(), copy(), dict(), json_schema(), json(), parse_obj(), update_forward_refs()
• [x] __fields__, __private_attributes__, __validators__
• [x] parse_raw is deprecated and should be replaced with model_validate_json
• [x] Pydantic 2 wants TypedDict to be imported from typing_extensions if using Python < 3.12, because it needs __orig_bases__ which 3.12 introduced
• [x] The validator migration does not recognize pydantic.validator etc.
• [x] Same for the Field migration
• [x] pydantic.Extra.forbid should be replaced with "forbid"
• [x] skip_on_failure=True should be removed from root_validator
• [x] Batch Pyre requests for speed
• [x] Should replace json.loads(x.model_dump_json()) with x.model_dump(mode="json").
• [x] parse_raw_as has been removed entirely. Use TypeAdapter.
• [x] parse_obj_as is not fully removed but should be replaced with TypeAdapter too.
• [x] In root_validators (→ model_validators) with mode="before", the second argument may be an instance of the actual model instead of a dict, in which case you should probably just return it. See example in docs. In fact it could be anything since you can pass anything to model_validate ! Add a TODO comment when migrating
• [x] model_validator(mode="after") needs to be an instance method, not a class method
• [x] For debugging, need a way to run bump-pydantic on a single file while still importing the class hierarchy from a full repo
• [x] Add a TODO/warning for BaseModel subclasses that override deprecated method like dict or json
• [x] If you have an Optional, in model.dict() if model else {} Pyre infers the first model as Optional even though None should be excluded there. work around.
• [x] If you have a field starting with model_, e.g. model_name, Pydantic 2 will complain. You need to add protected_namespaces=() to the ConfigDict
  • [x] model_config cannot be used as a field name at all
  • [x] we need to add a model_config if the original class had no Config
• [x] Sometimes it generates ConfigDict with two extra args
• [x] It does not handle __root__ with a type annotation
• [x] Some migrations break if you have nested BaseModel classes due to incorrect tracking logic.

Most of these changes should be generally useful: they migrate things that were not migrated, or they fix bugs in existing migrations. A few changes are needed to enable running bump_pydantic on a large repository. A couple of things depend on improvements to LibCST which I am also upstreaming.

I sent three of these as separate PRs about two months ago, but there has been no activity. Let's find a way to coordinate on how to get these changes upstreamed. I won't need this code again, but I'd like others to be able to benefit from it.

[povilasb]

That would be really useful for me too 🙇

[samuelcolvin]

Thanks so much for this. I'll get someone to look at it soon.