jsonargparse icon indicating copy to clipboard operation
jsonargparse copied to clipboard

Adding support for pydantic extras in models

Open ModdingFox opened this issue 6 months ago • 6 comments

This PR adds support for Pydantic models with extra field configuration (allow, forbid, ignore).

Key changes:

  • Models with extra="allow" now accept and include additional fields during parsing and instantiation
  • Models with extra="forbid" properly reject extra fields with clear error messages
  • Models with extra="ignore" accept extra fields during parsing but ignore them during instantiation
  • Support for both Pydantic v1 and v2 syntax and configuration styles

Behavior:

  • extra="allow": Extra fields are accepted and included in the final model
  • extra="forbid": Extra fields cause a NSKeyError during parsing
  • extra="ignore": Extra fields are accepted during parsing but ignored during model instantiation
  • Default behavior remains unchanged for models without explicit extra configuration

This resolves the limitation where jsonargparse would always reject extra fields regardless of the Pydantic model's extra configuration.

  • [x] Did you read the contributing guideline?
  • [ ] Did you update the documentation? (readme and public docstrings)
  • [x] Did you write unit tests such that there is 100% coverage on related code? (required for bug fixes and new features)
  • [x] Did you verify that new and existing tests pass locally?
  • [x] Did you make sure that all changes preserve backward compatibility?
  • [x] Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

ModdingFox avatar Jul 03 '25 06:07 ModdingFox

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 100.00%. Comparing base (6ad1dd6) to head (0940383). :warning: Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #732   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         6857      6895   +38     
=========================================
+ Hits          6857      6895   +38     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jul 06 '25 06:07 codecov[bot]

Oh wired the full pytest report had it at 100% locally. Will pull the pr back to fresh env and double check ^.^

ModdingFox avatar Jul 07 '25 15:07 ModdingFox

Hopefully this Dockerfile is useful to someone else.

FROM debian:bookworm

# System update and install build tools
RUN apt update
RUN apt upgrade -y
RUN apt install -y build-essential curl git

# Build user
RUN useradd -ms /bin/bash build
USER build
WORKDIR /home/build

# Mise setup
RUN curl https://mise.run | sh
ENV PATH="/home/build/.local/bin:$PATH"

# Python env setup
RUN mise install [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
RUN mise use -g [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
ENV PATH="/home/build/.local/share/mise/shims/:$PATH"

# Init build tool venv
RUN mise exec [email protected] -- python3 -m venv venv
ENV PATH="/home/build/venv/bin:$PATH"

# Clone repo for build and install build tools
RUN git clone https://github.com/ModdingFox/jsonargparse.git
# COPY --chown=build:build . .
# RUN git config --global --add safe.directory /home/build
WORKDIR /home/build/jsonargparse
RUN git checkout pydantic_allow_extra
RUN pip3 install -e ".[dev,all]"

# Setup git pre-commit hook
RUN pre-commit install

# Generate docs
RUN sphinx-build sphinx sphinx/_build sphinx/*.rst

# Run all tests
RUN rm -fr jsonargparse_tests/.coverage jsonargparse_tests/htmlcov
RUN tox -- --cov=../jsonargparse --cov-append
WORKDIR /home/build/jsonargparse/jsonargparse_tests
RUN coverage html

# Setup continer for exec access
WORKDIR /home/build/jsonargparse
ENTRYPOINT ["/bin/bash"]

ModdingFox avatar Jul 07 '25 21:07 ModdingFox

Following up here. Any additional change or comment to this pr?

ModdingFox avatar Jul 15 '25 20:07 ModdingFox

sweet appreciate the feedback. will look once i have a bit of time the rest of this month is gunna be stupid for me so ill likely look at addressing these in aug once i recover some spare brain cells ^.^

ModdingFox avatar Jul 17 '25 01:07 ModdingFox

@ModdingFox will you pick this up again soon? Note that the code changes so the longer it is, the more conflicts that will need to be fixed. And I am planning on doing more refactoring. If you don't have time for this now, maybe I help out to finish it off.

mauvilsa avatar Oct 23 '25 05:10 mauvilsa