python-sdk icon indicating copy to clipboard operation
python-sdk copied to clipboard

Improve test pyramid: strengthen unit tests alongside integration tests

Open dgenio opened this issue 1 month ago • 0 comments

Description

Summary

The current test suite for the Python SDK appears to lean heavily on integration tests:

  • many tests spin up full servers/clients,
  • numerous tests live under tests/issues/ as end-to-end regressions.

These are valuable, but they make it harder to:

  • get fast feedback from focused unit tests,
  • refactor internal components without breaking many tests at once,
  • pinpoint which component is responsible when a regression occurs.

Proposal

  1. Identify core units suitable for isolated testing

    Examples might include:

    • schema generation and func_metadata() helpers,
    • message parsing/serialization,
    • specific error mapping logic,
    • pure utility modules.
  2. Add targeted unit tests

    • For each such module, add tests that do not require spinning up a full session or transport.
    • Use mocks/fakes at the boundaries to keep tests small and fast.
  3. Keep integration tests, but make them more focused

    • Retain high-value end-to-end tests, especially for bug regressions and cross-component behavior.
    • When possible, scope regression tests to the minimal surface needed to reproduce the bug.

Why this matters

  • Developer velocity: Quicker feedback when changing internals.
  • Refactorability: Internal improvements (e.g., transport abstraction, session state machine) become safer.
  • Signal quality: When tests fail, it’s easier to see which component is responsible.

Acceptance criteria

  • [ ] Key modules (e.g., schema generation, error mapping) have unit tests that do not depend on full server/client setup.
  • [ ] Integration tests still cover end-to-end behavior and known regressions.
  • [ ] New contributions are encouraged to add unit tests where appropriate, not only integration tests.

References

No response

dgenio avatar Nov 28 '25 15:11 dgenio