datamodel-code-generator icon indicating copy to clipboard operation
datamodel-code-generator copied to clipboard

Too short timeout for fetching schema

Open azatoth opened this issue 1 year ago • 0 comments

Describe the bug The current timeout for when fetching a schema from remove host can sometime be way too short and resulting in the application fails over and over.

To Reproduce Used commandline:

$ datamodel-codegen --url "https://lldev.thespacedevs.com/2.3.0/schema" --input-file-type openapi --output "src/thespacedevs_api.py" --snake-case-field --use-default --force-optional

most often than not drops a python exception slug with httpx.ReadTimeout: The read operation timed out as the error:

Traceback

Traceback (most recent call last):
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
    yield
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 233, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    return self._connection.handle_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 143, in handle_request
    raise exc
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 113, in handle_request
    ) = self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 186, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 224, in _receive_event
    data = self._network_stream.read(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_backends/sync.py", line 124, in read
    with map_exceptions(exc_map):
  File "/home/azatoth/.pyenv/versions/3.11.0/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ReadTimeout: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/datamodel_code_generator/__main__.py", line 449, in main
    generate(
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/datamodel_code_generator/__init__.py", line 314, in generate
    input_text = remote_text_cache.get_or_put(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/datamodel_code_generator/parser/__init__.py", line 28, in get_or_put
    value = self[key] = default_factory(key)
                        ^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/datamodel_code_generator/__init__.py", line 316, in <lambda>
    default_factory=lambda url: get_body(
                                ^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/datamodel_code_generator/http.py", line 19, in get_body
    return httpx.get(
           ^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_api.py", line 198, in get
    return request(
           ^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_api.py", line 106, in request
    return client.request(
           ^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 827, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1015, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "/home/azatoth/.pyenv/versions/3.11.0/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/azatoth/.cache/pypoetry/virtualenvs/labpadre-assistant-fK4b4gu4-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout: The read operation timed out

Expected behavior It should wait at least a reasonable amount of time before gracefully giving up

Version:

  • OS: linux
  • Python version: 3.11
  • datamodel-code-generator version: 0.25.8

azatoth avatar Aug 17 '24 13:08 azatoth