marshmallow-recipe icon indicating copy to clipboard operation
marshmallow-recipe copied to clipboard

Slim dump/load implementation

Open Pliner opened this issue 7 months ago • 0 comments

TLDR: slim methods speed up transaction_load_dump.py from 0.019 seconds to 0.003 seconds.

Slim version:

 7067 function calls in 0.003 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      128    0.001    0.000    0.002    0.000 serialization_slim.py:291(load)
      128    0.000    0.000    0.001    0.000 serialization_slim.py:269(dump)
      256    0.000    0.000    0.000    0.000 serialization_slim.py:244(load)
      256    0.000    0.000    0.000    0.000 serialization_slim.py:236(dump)
      128    0.000    0.000    0.000    0.000 uuid.py:139(__init__)
      128    0.000    0.000    0.000    0.000 {method 'isoformat' of 'datetime.datetime' objects}
      128    0.000    0.000    0.000    0.000 <string>:2(__init__)
      128    0.000    0.000    0.000    0.000 uuid.py:280(__str__)
     1284    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
      128    0.000    0.000    0.000    0.000 serialization_slim.py:157(load)
      512    0.000    0.000    0.000    0.000 {method 'quantize' of 'decimal.Decimal' objects}
      128    0.000    0.000    0.000    0.000 serialization_slim.py:152(dump)
        1    0.000    0.000    0.002    0.002 serialization_slim.py:112(load)
        1    0.000    0.000    0.001    0.001 serialization_slim.py:93(dump)
      128    0.000    0.000    0.000    0.000 serialization_slim.py:216(load)
      128    0.000    0.000    0.000    0.000 serialization_slim.py:211(dump)
      642    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
      384    0.000    0.000    0.000    0.000 {method 'replace' of 'str' objects}
      640    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
        1    0.000    0.000    0.003    0.003 {built-in method builtins.exec}
      256    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
      512    0.000    0.000    0.000    0.000 {method 'is_nan' of 'decimal.Decimal' objects}
      512    0.000    0.000    0.000    0.000 {method 'is_infinite' of 'decimal.Decimal' objects}
        1    0.000    0.000    0.003    0.003 <string>:1(<module>)
      128    0.000    0.000    0.000    0.000 {built-in method fromisoformat}
      128    0.000    0.000    0.000    0.000 {method 'count' of 'list' objects}
      128    0.000    0.000    0.000    0.000 {method 'strip' of 'str' objects}
      130    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.001    0.001 serialization_slim.py:29(dump_slim_many)
        2    0.000    0.000    0.000    0.000 serialization_slim.py:328(__get_field_for_dataclass)
        2    0.000    0.000    0.000    0.000 dataclasses.py:1256(is_dataclass)
        1    0.000    0.000    0.002    0.002 serialization_slim.py:54(load_slim_many)
        2    0.000    0.000    0.000    0.000 <string>:2(__hash__)
        2    0.000    0.000    0.000    0.000 <string>:2(__eq__)
        2    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.hash}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Baseline:

 35400 function calls (35016 primitive calls) in 0.019 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    258/2    0.002    0.000    0.014    0.007 schema.py:576(_deserialize)
     1280    0.001    0.000    0.008    0.000 fields.py:344(deserialize)
      768    0.001    0.000    0.001    0.000 fields.py:1093(_format_num)
     1024    0.001    0.000    0.002    0.000 fields.py:265(_validate)
     1280    0.001    0.000    0.008    0.000 schema.py:655(getter)
     1024    0.001    0.000    0.001    0.000 fields.py:271(_validate_all)
      640    0.001    0.000    0.003    0.000 fields.py:317(serialize)
      256    0.001    0.000    0.001    0.000 uuid.py:139(__init__)
      512    0.001    0.000    0.002    0.000 fields.py:1103(_validated)
    129/1    0.001    0.000    0.004    0.004 schema.py:501(_serialize)
     1280    0.001    0.000    0.009    0.000 schema.py:481(_call_and_store)
        2    0.000    0.000    0.014    0.007 schema.py:611(<listcomp>)
      640    0.000    0.000    0.001    0.000 utils.py:262(get_value)
      256    0.000    0.000    0.001    0.000 inspect.py:391(_has_code_flag)
     2562    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
     1024    0.000    0.000    0.000    0.000 validate.py:66(__init__)
     1280    0.000    0.000    0.000    0.000 utils.py:298(set_value)
      640    0.000    0.000    0.000    0.000 utils.py:288(_get_value_for_key)
        4    0.000    0.000    0.001    0.000 schema.py:1221(<listcomp>)
     1024    0.000    0.000    0.000    0.000 validate.py:73(__call__)
      256    0.000    0.000    0.001    0.000 fields.py:912(_validated)
      768    0.000    0.000    0.000    0.000 {method 'quantize' of 'decimal.Decimal' objects}
      256    0.000    0.000    0.001    0.000 fields.py:673(_deserialize)
      128    0.000    0.000    0.000    0.000 {method 'isoformat' of 'datetime.datetime' objects}
     1280    0.000    0.000    0.000    0.000 fields.py:308(_validate_missing)
      640    0.000    0.000    0.001    0.000 fields.py:253(get_value)
      512    0.000    0.000    0.001    0.000 fields.py:952(_validated)
      512    0.000    0.000    0.002    0.000 fields.py:976(_deserialize)
      640    0.000    0.000    0.001    0.000 schema.py:469(get_attribute)
      128    0.000    0.000    0.000    0.000 uuid.py:280(__str__)
      256    0.000    0.000    0.000    0.000 fields.py:1285(_deserialize)
      256    0.000    0.000    0.001    0.000 utils.py:46(is_generator)
     1670    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
      256    0.000    0.000    0.001    0.000 fields.py:969(_serialize)
      258    0.000    0.000    0.001    0.000 utils.py:51(is_iterable_but_not_string)
      768    0.000    0.000    0.000    0.000 {method 'replace' of 'str' objects}
      256    0.000    0.000    0.000    0.000 fields.py:681(_serialize)
      258    0.000    0.000    0.001    0.000 utils.py:56(is_collection)
      256    0.000    0.000    0.001    0.000 inspect.py:402(isgeneratorfunction)
      256    0.000    0.000    0.001    0.000 fields.py:925(_deserialize)
      900    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
      128    0.000    0.000    0.000    0.000 bake.py:287(<dictcomp>)
      256    0.000    0.000    0.000    0.000 inspect.py:378(isfunction)
      128    0.000    0.000    0.000    0.000 fields.py:1276(_serialize)
      128    0.000    0.000    0.000    0.000 bake.py:290(post_load)
      257    0.000    0.000    0.000    0.000 typing.py:359(inner)
      256    0.000    0.000    0.000    0.000 inspect.py:2077(_signature_is_functionlike)
      256    0.000    0.000    0.000    0.000 functools.py:421(_unwrap_partial)
      128    0.000    0.000    0.000    0.000 bake.py:284(remove_none_values)
      256    0.000    0.000    0.000    0.000 inspect.py:300(ismethod)
      128    0.000    0.000    0.000    0.000 utils.py:242(ensure_text_type)
      258    0.000    0.000    0.000    0.000 <frozen abc>:117(__instancecheck__)
      256    0.000    0.000    0.000    0.000 inspect.py:428(isgenerator)
      256    0.000    0.000    0.000    0.000 {built-in method builtins.format}
      768    0.000    0.000    0.000    0.000 {method 'is_finite' of 'decimal.Decimal' objects}
      256    0.000    0.000    0.000    0.000 bake.py:294(pre_load)
      384    0.000    0.000    0.000    0.000 schema.py:414(dict_class)
      256    0.000    0.000    0.000    0.000 fields.py:1113(_to_string)
      644    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
      513    0.000    0.000    0.000    0.000 typing.py:2268(cast)
      258    0.000    0.000    0.000    0.000 {built-in method _abc._abc_instancecheck}
      256    0.000    0.000    0.000    0.000 {built-in method fromisoformat}
      128    0.000    0.000    0.000    0.000 fields.py:892(_serialize)
        2    0.000    0.000    0.014    0.007 schema.py:811(_do_load)
      512    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
      512    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
      512    0.000    0.000    0.000    0.000 {method 'is_infinite' of 'decimal.Decimal' objects}
      512    0.000    0.000    0.000    0.000 {method 'is_nan' of 'decimal.Decimal' objects}
        1    0.000    0.000    0.004    0.004 schema.py:512(<listcomp>)
      256    0.000    0.000    0.000    0.000 {method 'count' of 'list' objects}
      256    0.000    0.000    0.000    0.000 {method 'strip' of 'str' objects}
        1    0.000    0.000    0.019    0.019 {built-in method builtins.exec}
      256    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.019    0.019 <string>:1(<module>)
        1    0.000    0.000    0.004    0.004 schema.py:525(dump)
        1    0.000    0.000    0.007    0.007 schema.py:779(validate)
        1    0.000    0.000    0.012    0.012 serialization.py:91(dump_many_v3)
        8    0.000    0.000    0.001    0.000 schema.py:1196(_invoke_processors)
        3    0.000    0.000    0.001    0.000 schema.py:1077(_invoke_load_processors)
        2    0.000    0.000    0.000    0.000 serialization.py:57(schema_v3)
        1    0.000    0.000    0.007    0.007 serialization.py:73(load_many_v3)
        2    0.000    0.000    0.000    0.000 schema.py:1106(_invoke_field_validators)
        1    0.000    0.000    0.000    0.000 schema.py:1063(_invoke_dump_processors)
        2    0.000    0.000    0.000    0.000 <string>:2(__init__)
        1    0.000    0.000    0.007    0.007 schema.py:692(load)
        2    0.000    0.000    0.000    0.000 <string>:2(__hash__)
        2    0.000    0.000    0.000    0.000 <string>:2(__eq__)
        2    0.000    0.000    0.000    0.000 error_store.py:13(__init__)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.hash}

Pliner avatar Jan 15 '24 07:01 Pliner