The yaml loading is slow when reading big openapi files (3MB+). Especially while debugging.

[Wim-De-Clercq]

The main reason there is a custom loader in this library is to replace integer keys into strings. However, the json library from python could handle this for you.

It is in fact faster to use the original yaml loaders in combination with json dumping and loading than using a custom loader. And if the LibYAML package is installed it's even factor 9-10 faster.

(note you have to edit the paths if you want to run the below)

import json

from yaml import SafeLoader
from yaml import CSafeLoader
from yaml import load

from openapi_spec_validator.loaders import ExtendedSafeLoader


def read_yaml_file(path, loader=ExtendedSafeLoader):
    """Open a file, read it and return its contents."""
    with open(path) as fh:
        return load(fh, loader)


def read_yaml_file_fast(path, loader=SafeLoader):
    """Open a file, read it and return its contents."""
    with open(path) as fh:
        return json.loads(json.dumps(load(fh, loader)))


def read_yaml_file_faster(path, loader=CSafeLoader):
    """Open a file, read it and return its contents."""
    with open(path) as fh:
        return json.loads(json.dumps(load(fh, loader)))


if __name__ == '__main__':

    import timeit
    result = timeit.timeit(
        "read_yaml_file('EDIT/openapi-spec-validator/tests/integration/data/v3.1/petstore.yaml')",
        "from __main__ import read_yaml_file",
        number=1000
    )
    print("original:", result)
    result = timeit.timeit(
        "read_yaml_file_fast('EDIT/openapi-spec-validator/tests/integration/data/v3.1/petstore.yaml')",
        "from __main__ import read_yaml_file_fast",
        number=1000
    )
    print("+json:", result)
    result = timeit.timeit(
        "read_yaml_file_faster('EDIT/openapi-spec-validator/tests/integration/data/v3.1/petstore.yaml')",
        "from __main__ import read_yaml_file_faster",
        number=1000
    )
    print("cloader + json:", result)

original: 9.896625495999615
+json: 9.384017411000968
cloader + json: 1.066654717998972

My 3+MB file takes 9.2 seconds to load with the original code. Versus 1,5seconds with the Cloader. And with debugging mode on, this becomes 40+ seconds with original versus 4 seconds with Cloader.

See PR: https://github.com/p1c2u/openapi-spec-validator/pull/146