Feat/record and play framework
Before submitting this PR, please make sure:
- [x] Your code builds clean without any errors or warnings
- [x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")
A description of the changes proposed in the pull request:
-
New Features
- Introduced a comprehensive "Record and Play" (RnP) testing framework with documentation, a reusable test harness, and dependency mocking strategies.
- Added modular Fastify API endpoints and a sample API demonstrating diverse dependency scenarios.
- Implemented snapshot-based API test cases with mock data management ensuring deterministic unit tests.
- Provided custom serialization support for complex data types like BigNumber in tests.
- Added VS Code configuration guide for managing multiple Jest test suites and debugging workflows.
-
Bug Fixes
- Updated test cases to expect correct HTTP status codes (400 instead of 500) for invalid input scenarios.
-
Chores
- Enhanced test scripts, configuration, and environment variable handling for improved test isolation and reproducibility.
- Added new test setup and configuration files to support the RnP framework and snapshot testing.
-
Documentation
- Added a detailed guide explaining the RnP testing workflow, conventions, and best practices.
Tests performed by the developer:
pnpm test-record (changes files intentionally)
pnpm test rnp
Tips for QA testing:
- run
pnpm test- verify that all tests pass - run
pnpm test rnp- verify that only tests with rnp in the name are run and pass - run
pnpm test-play- verify that only play tests are run and pass - run
pnpm test-record- verify that rnpExample mock files and snapshot file changes change - run
pnpm test-play- verify that only rnp tests are run and pass - revert changes to rnpExample.test.ts.snap
- run
pnpm test-play- verify that rnpExample play tests relying on mocks fail - revert all changes to mock files
- run
pnpm test-play- verify that tests pass
Separated the "Play" tests from the primary test suite today with https://github.com/hummingbot/gateway/pull/490/commits/9bd329a5e0bf7810321a232fcae0a7d3cb721ed2 as LLM agents were running the unit test suite with -u to force snapshots to avoid having to actually debug issues. Also clarified everywhere "Record" and "Play" Tests rather than saying unit.
I recommend doing a squash merge when eventually merging this as 42 commits is pretty outrageous for the functionality being added. However, I think keeping the git history in the PR is important for posterity and to understand why some design decisions were made.