datamodel-code-generator icon indicating copy to clipboard operation
datamodel-code-generator copied to clipboard

Run test without fix - pydantic 1.9.1

Open eyal-mor opened this issue 2 years ago • 0 comments

Fix for #810 due to deep copy introduction in Pydantic.

It seems like pydantic 1.9.1 added deepcopy to model validation:

Config.copy_on_model_validation does a deep copy and not a shallow one, https://github.com/samuelcolvin/pydantic/issues/3641

(from the release notes)

Current tests fail with a simple poetry upgrade.

poetry run ./scripts/test.sh
================================================================================================ test session starts ================================================================================================
platform darwin -- Python 3.10.3, pytest-7.0.1, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: datamodel-code-generator, configfile: pyproject.toml
plugins: benchmark-3.4.1, mock-3.6.1, cov-3.0.0
collected 343 items                                                                                                                                                                                                 

tests/test_format.py .                                                                                                                                                                                        [  0%]
tests/test_imports.py .......                                                                                                                                                                                 [  2%]
tests/test_main.py .................F.......................................................................................................s..................                                               [ 43%]
tests/test_main_kr.py ..........                                                                                                                                                                              [ 46%]
tests/test_reference.py ..................                                                                                                                                                                    [ 51%]
tests/model/test_base.py ....                                                                                                                                                                                 [ 52%]
tests/model/pydantic/test_base_model.py .....................                                                                                                                                                 [ 58%]
tests/model/pydantic/test_custom_root_type.py ....                                                                                                                                                            [ 59%]
tests/model/pydantic/test_data_class.py ....                                                                                                                                                                  [ 60%]
tests/model/pydantic/test_types.py ........................................                                                                                                                                   [ 72%]
tests/parser/test_base.py .................                                                                                                                                                                   [ 77%]
tests/parser/test_jsonschema.py ..........................................                                                                                                                                    [ 89%]
tests/parser/test_openapi.py ...................................                                                                                                                                              [100%]

===================================================================================================== FAILURES ======================================================================================================
___________________________________________________________________________________________ test_main_modular_reuse_model ___________________________________________________________________________________________

tmpdir_factory = TempdirFactory(_tmppath_factory=TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x104193730>, _basetemp=PosixPath('/private/var/folders/ws/znl0s07x0hx7cbflpd52tblc0000gn/T/pytest-of-eyalmor/pytest-3')))

    def test_main_modular_reuse_model(tmpdir_factory: TempdirFactory) -> None:
        """Test main function on modular file."""
    
        output_directory = Path(tmpdir_factory.mktemp('output'))
    
        input_filename = OPEN_API_DATA_PATH / 'modular.yaml'
        output_path = output_directory / 'model'
    
        with freeze_time(TIMESTAMP):
            main(
                [
                    '--input',
                    str(input_filename),
                    '--output',
                    str(output_path),
                    '--reuse-model',
                ]
            )
        main_modular_dir = EXPECTED_MAIN_PATH / 'main_modular_reuse_model'
        for path in main_modular_dir.rglob('*.py'):
>           result = output_path.joinpath(path.relative_to(main_modular_dir)).read_text()

tests/test_main.py:475: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../.pyenv/versions/3.10.3/lib/python3.10/pathlib.py:1132: in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = PosixPath('/private/var/folders/ws/znl0s07x0hx7cbflpd52tblc0000gn/T/pytest-of-eyalmor/pytest-3/output1/model/models.py'), mode = 'r', buffering = -1, encoding = 'locale', errors = None, newline = None

    def open(self, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None):
        """
        Open the file pointed by this path and return a file object, as
        the built-in open() function does.
        """
        if "b" not in mode:
            encoding = io.text_encoding(encoding)
>       return self._accessor.open(self, mode, buffering, encoding, errors,
                                   newline)
E       FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/ws/znl0s07x0hx7cbflpd52tblc0000gn/T/pytest-of-eyalmor/pytest-3/output1/model/models.py'

../../../.pyenv/versions/3.10.3/lib/python3.10/pathlib.py:1117: FileNotFoundError
----------------------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "datamodel-code-generator/datamodel_code_generator/__main__.py", line 541, in main
    generate(
  File "datamodel-code-generator/datamodel_code_generator/__init__.py", line 367, in generate
    results = parser.parse()
  File "datamodel-code-generator/datamodel_code_generator/parser/base.py", line 634, in parse
    inherited_model = model.__class__(
  File "datamodel-code-generator/datamodel_code_generator/model/pydantic/base_model.py", line 152, in __init__
    super().__init__(
  File "datamodel-code-generator/datamodel_code_generator/model/base.py", line 188, in __init__
    self.base_classes: List[BaseClassDataType] = [
  File "datamodel-code-generator/datamodel_code_generator/model/base.py", line 189, in <listcomp>
    BaseClassDataType(reference=b) for b in base_classes
  File "datamodel-code-generator/datamodel_code_generator/types.py", line 210, in __init__
    super().__init__(**values)
  File "datamodel-code-generator/datamodel_code_generator/reference.py", line 45, in __init__
    super().__init__(**values)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for BaseClassDataType
reference
  Template.__new__() missing 1 required positional argument: 'source' (type=type_error)

================================================================================================= warnings summary ==================================================================================================
tests/test_main.py::test_pyproject
tests/test_main.py::test_validation
  datamodel-code-generator/.venv/lib/python3.10/site-packages/prance/__init__.py:173: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    parsed = distutils.version.StrictVersion(spec_version).version

tests/parser/test_jsonschema.py::test_get_data_type[string-unknown-type-str-None-None]
  datamodel-code-generator/datamodel_code_generator/parser/jsonschema.py:262: UserWarning: format of 'unknown-type' not understood for 'string' - using default
    warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform darwin, python 3.10.3-final-0 ----------
Name                                                          Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------------------
datamodel_code_generator/__init__.py                            161     13     58      0    93%   39-40, 43-44, 52-64
datamodel_code_generator/__main__.py                            211      0     59      0   100%
datamodel_code_generator/format.py                               53      4     14      2    91%   54, 109, 129-130
datamodel_code_generator/http.py                                  7      0      0      0   100%
datamodel_code_generator/imports.py                              56      0     22      0   100%
datamodel_code_generator/model/__init__.py                        2      0      0      0   100%
datamodel_code_generator/model/base.py                          168      0     54      0   100%
datamodel_code_generator/model/enum.py                           37      0     10      0   100%
datamodel_code_generator/model/pydantic/__init__.py              15      0      4      0   100%
datamodel_code_generator/model/pydantic/base_model.py            94      2     51      1    98%   181-182
datamodel_code_generator/model/pydantic/custom_root_type.py       5      0      2      0   100%
datamodel_code_generator/model/pydantic/dataclass.py              7      0      2      0   100%
datamodel_code_generator/model/pydantic/imports.py               34      0      0      0   100%
datamodel_code_generator/model/pydantic/types.py                109      0     72      0   100%
datamodel_code_generator/parser/__init__.py                      15      0      4      0   100%
datamodel_code_generator/parser/base.py                         314      0    191      0   100%
datamodel_code_generator/parser/jsonschema.py                   546      2    293      2    99%   643, 847
datamodel_code_generator/parser/openapi.py                      163      0     68      0   100%
datamodel_code_generator/reference.py                           315      0    134      0   100%
datamodel_code_generator/types.py                               193      1     80      0    99%   370
datamodel_code_generator/version.py                               1      0      0      0   100%
---------------------------------------------------------------------------------------------------------
TOTAL                                                          2506     22   1118      5    99%
Coverage XML written to file coverage.xml

============================================================================================== short test summary info ==============================================================================================
FAILED tests/test_main.py::test_main_modular_reuse_model - FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/ws/znl0s07x0hx7cbflpd52tblc0000gn/T/pytest-of-eyalmor/pytest-3/output1/mo...
=============================================================================== 1 failed, 341 passed, 1 skipped, 3 warnings in 11.74s ===============================================================================

However, with the fix, the tests pass successfully:

poetry run ./scripts/test.sh
================================================================================================ test session starts ================================================================================================
platform darwin -- Python 3.10.3, pytest-7.0.1, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: datamodel-code-generator, configfile: pyproject.toml
plugins: benchmark-3.4.1, mock-3.6.1, cov-3.0.0
collected 343 items                                                                                                                                                                                                 

tests/test_format.py .                                                                                                                                                                                        [  0%]
tests/test_imports.py .......                                                                                                                                                                                 [  2%]
tests/test_main.py .........................................................................................................................s..................                                               [ 43%]
tests/test_main_kr.py ..........                                                                                                                                                                              [ 46%]
tests/test_reference.py ..................                                                                                                                                                                    [ 51%]
tests/model/test_base.py ....                                                                                                                                                                                 [ 52%]
tests/model/pydantic/test_base_model.py .....................                                                                                                                                                 [ 58%]
tests/model/pydantic/test_custom_root_type.py ....                                                                                                                                                            [ 59%]
tests/model/pydantic/test_data_class.py ....                                                                                                                                                                  [ 60%]
tests/model/pydantic/test_types.py ........................................                                                                                                                                   [ 72%]
tests/parser/test_base.py .................                                                                                                                                                                   [ 77%]
tests/parser/test_jsonschema.py ..........................................                                                                                                                                    [ 89%]
tests/parser/test_openapi.py ...................................                                                                                                                                              [100%]

================================================================================================= warnings summary ==================================================================================================
tests/test_main.py::test_pyproject
tests/test_main.py::test_validation
  datamodel-code-generator/.venv/lib/python3.10/site-packages/prance/__init__.py:173: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    parsed = distutils.version.StrictVersion(spec_version).version

tests/parser/test_jsonschema.py::test_get_data_type[string-unknown-type-str-None-None]
  datamodel-code-generator/datamodel_code_generator/parser/jsonschema.py:262: UserWarning: format of 'unknown-type' not understood for 'string' - using default
    warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform darwin, python 3.10.3-final-0 ----------
Name                                                          Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------------------
datamodel_code_generator/__init__.py                            161     13     58      0    93%   39-40, 43-44, 52-64
datamodel_code_generator/__main__.py                            211      0     59      0   100%
datamodel_code_generator/format.py                               53      4     14      2    91%   54, 109, 129-130
datamodel_code_generator/http.py                                  7      0      0      0   100%
datamodel_code_generator/imports.py                              56      0     22      0   100%
datamodel_code_generator/model/__init__.py                        2      0      0      0   100%
datamodel_code_generator/model/base.py                          168      0     54      0   100%
datamodel_code_generator/model/enum.py                           37      0     10      0   100%
datamodel_code_generator/model/pydantic/__init__.py              15      0      4      0   100%
datamodel_code_generator/model/pydantic/base_model.py            94      2     51      1    98%   181-182
datamodel_code_generator/model/pydantic/custom_root_type.py       5      0      2      0   100%
datamodel_code_generator/model/pydantic/dataclass.py              7      0      2      0   100%
datamodel_code_generator/model/pydantic/imports.py               34      0      0      0   100%
datamodel_code_generator/model/pydantic/types.py                109      0     72      0   100%
datamodel_code_generator/parser/__init__.py                      15      0      4      0   100%
datamodel_code_generator/parser/base.py                         314      0    191      0   100%
datamodel_code_generator/parser/jsonschema.py                   546      2    293      2    99%   643, 847
datamodel_code_generator/parser/openapi.py                      163      0     68      0   100%
datamodel_code_generator/reference.py                           316      0    134      0   100%
datamodel_code_generator/types.py                               194      1     80      0    99%   370
datamodel_code_generator/version.py                               1      0      0      0   100%
---------------------------------------------------------------------------------------------------------
TOTAL                                                          2508     22   1118      5    99%
Coverage XML written to file coverage.xml

==================================================================================== 342 passed, 1 skipped, 3 warnings in 8.43s =====================================================================================

eyal-mor avatar Aug 10 '22 12:08 eyal-mor

Codecov Report

Merging #819 (83a0e71) into master (e9b6edf) will not change coverage. The diff coverage is 100.00%.

@@             Coverage Diff             @@
##            master      #819     +/-   ##
===========================================
  Coverage   100.00%   100.00%             
===========================================
  Files           11        21     +10     
  Lines         1020      2511   +1491     
  Branches       201       556    +355     
===========================================
+ Hits          1020      2511   +1491     
Flag Coverage Δ
unittests 99.92% <99.89%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
datamodel_code_generator/model/__init__.py 100.00% <ø> (ø)
datamodel_code_generator/__init__.py 100.00% <100.00%> (ø)
datamodel_code_generator/__main__.py 100.00% <100.00%> (ø)
datamodel_code_generator/format.py 100.00% <100.00%> (ø)
datamodel_code_generator/http.py 100.00% <100.00%> (ø)
datamodel_code_generator/imports.py 100.00% <100.00%> (ø)
datamodel_code_generator/model/base.py 100.00% <100.00%> (ø)
datamodel_code_generator/model/enum.py 100.00% <100.00%> (ø)
...atamodel_code_generator/model/pydantic/__init__.py 100.00% <100.00%> (ø)
...amodel_code_generator/model/pydantic/base_model.py 100.00% <100.00%> (ø)
... and 18 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Aug 11 '22 16:08 codecov[bot]

@eyalmor-ent I have released the PR as 0.13.1 Thank you very much 🙇

koxudaxi avatar Aug 11 '22 18:08 koxudaxi