langchain
langchain copied to clipboard
langchain-ai
make running specific unit tests easier
I find it's easier to do TDD if i can run specific unit tests. I know watch is there but some people prefer running their tests manually.
Hi @rick2047 thank you for the contribution! :wave:
I'm not super familiar with Makefile syntax. I patched the PR in and haven't been able to run tests selectively
langchain git:(make_test_args) ✗ make test tests/unit_tests/output_parsers/test_regex_dict.py
runs all unit tests
If you're making the update could you also update both test and tests entry points?
Ok I may have biffed my last commit. It should work now and the doc string is correct now. So for example we can now do,
(langchain) rick@MSI:~/langchain$ make test test_file=tools/python/test_python.py
poetry run pytest tests/unit_tests/tools/python/test_python.py
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/rick/langchain
plugins: asyncio-0.20.3, anyio-3.6.2, cov-4.0.0, dotenv-0.5.2, mock-3.10.0
asyncio: mode=strict
collected 2 items
tests/unit_tests/tools/python/test_python.py .. [100%]
============================================================================ 2 passed in 0.71s ============================================================================
And both test and tests entry points are updated. I don't know why they are two different things though, they are running the exact same command. And tests doesn't even have a help entry.
I realized as a side effect I can also do this to run all tests in a folder. It's recursive too.
(langchain) rick@MSI:~/langchain$ make test test_file=tools/
poetry run pytest tests/unit_tests/tools
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/rick/langchain
plugins: asyncio-0.20.3, anyio-3.6.2, cov-4.0.0, dotenv-0.5.2, mock-3.10.0
asyncio: mode=strict
collected 180 items
tests/unit_tests/tools/test_base.py ........................ [ 13%]
tests/unit_tests/tools/test_exported.py . [ 13%]
tests/unit_tests/tools/test_json.py .... [ 16%]
tests/unit_tests/tools/test_public_api.py . [ 16%]
tests/unit_tests/tools/test_signatures.py ............................................................ [ 50%]
tests/unit_tests/tools/file_management/test_copy.py ... [ 51%]
tests/unit_tests/tools/file_management/test_file_search.py ... [ 53%]
tests/unit_tests/tools/file_management/test_list_dir.py ... [ 55%]
tests/unit_tests/tools/file_management/test_move.py ... [ 56%]
tests/unit_tests/tools/file_management/test_read.py .. [ 57%]
tests/unit_tests/tools/file_management/test_toolkit.py .... [ 60%]
tests/unit_tests/tools/file_management/test_utils.py ..... [ 62%]
tests/unit_tests/tools/file_management/test_write.py ... [ 64%]
tests/unit_tests/tools/openapi/test_api_models.py ................................................... [ 92%]
tests/unit_tests/tools/python/test_python.py .. [ 93%]
tests/unit_tests/tools/requests/test_tool.py ...... [ 97%]
tests/unit_tests/tools/shell/test_shell.py ..... [100%]
============================================================================ warnings summary =============================================================================
tests/unit_tests/tools/shell/test_shell.py::test_shell_input_validation
/home/rick/langchain/langchain/tools/shell/tool.py:33: UserWarning: The shell tool has no safeguards by default. Use at your own risk.
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================== 180 passed, 1 warning in 1.06s ======================================================================
@rick2047 are you able to make the path to not be specified relative to test/unit_tests/ so that a user could use the shell to auto-complete paths with tab complete?
@eyurtsev This is what you were talking about?
Also by capitalizing the TEST_FILE variable in the file the command changed a bit (we have to capitalize the TEST_FILE on CLI). Now it looks like this
(langchain) rick@MSI:~/langchain$ make test TEST_FILE=tests/unit_tests/tools/python
poetry run pytest tests/unit_tests/tools/python
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/rick/langchain
plugins: asyncio-0.20.3, anyio-3.6.2, cov-4.0.0, dotenv-0.5.2, mock-3.10.0
asyncio: mode=strict
collected 2 items
tests/unit_tests/tools/python/test_python.py .. [100%]
============================================================================ 2 passed in 1.09s ============================================================================