Set up Python testing infrastructure with Poetry and pytest

[llbbl]

Set Up Python Testing Infrastructure

Summary

This PR establishes a comprehensive testing infrastructure for the project using Poetry as the package manager and pytest as the testing framework. The setup provides a solid foundation for writing and maintaining tests with proper organization, coverage reporting, and development tooling.

Changes Made

Package Management

• Poetry Configuration: Set up Poetry as the primary package manager with pyproject.toml
• Dependency Migration: Migrated existing dependencies from requirements.txt to Poetry
• Optional Dependencies: Configured PyQt5 as an optional dependency group due to platform-specific installation challenges

Testing Framework

• pytest: Configured as the main testing framework with comprehensive settings
• pytest-cov: Added for coverage reporting with HTML and XML output formats
• pytest-mock: Included for mocking utilities in tests

Project Structure

tests/
├── __init__.py
├── conftest.py         # Shared fixtures and configuration
├── test_validation.py  # Infrastructure validation tests
├── unit/              # Unit tests directory
│   └── __init__.py
└── integration/       # Integration tests directory
    └── __init__.py

Configuration Details

pytest Configuration (in pyproject.toml)

• Test discovery patterns configured
• Coverage reporting with multiple formats (terminal, HTML, XML)
• Custom markers: unit, integration, slow
• Strict mode enabled for better error detection
• Coverage threshold currently set to 0% (with TODO to change to 80% when adding real tests)

Shared Fixtures (conftest.py)

• temp_dir: Temporary directory for test files
• mock_config: Mock configuration object
• mock_config_file: Temporary config file creation
• mock_browser: Mock browser object for UI testing
• mock_requests_session: Mock HTTP session
• sample_html and mock_pyquery: HTML testing utilities
• reset_environment: Environment variable isolation
• capture_logs: Log capture for testing

Development Commands

• poetry run test - Run all tests
• poetry run tests - Alternative command (both work)
• All standard pytest options are available

Testing the Infrastructure

The setup includes validation tests that verify:

• Python version compatibility (3.8+)
• Project structure integrity
• Fixture availability and functionality
• Custom marker configuration
• Coverage setup
• Module imports

Run the validation tests:

poetry install
poetry run test

Notes

1. PyQt5 Dependency: Due to platform-specific installation issues on Linux, PyQt5 has been moved to an optional dependency group. Install with GUI support using:

   poetry install --with gui
   

2. Coverage Threshold: The coverage threshold is temporarily set to 0% to allow the validation tests to pass. This should be changed to 80% (as noted in the TODO comment) once actual unit tests are added.

3. No Production Code Tests: This PR only sets up the testing infrastructure. Actual unit and integration tests for the application code should be added in subsequent PRs.

Next Steps

1. Change coverage threshold from 0% to 80% in pyproject.toml
2. Write unit tests for core modules (config, browser, job modules)
3. Add integration tests for job workflows
4. Set up CI/CD pipeline to run tests automatically
5. Consider adding additional testing tools (e.g., pytest-asyncio for async code, pytest-timeout for test timeouts)