[chore] Transition linter execution from local venv to uvx-driven

Open jonpspri opened this issue 1 month ago • 0 comments

Transition linter execution from local venv to uvx-driven

Summary

Replace venv-based Python linter execution in the Makefile with uvx (uv's tool runner) to improve consistency, reduce venv bloat, and simplify dependency management. This change will make linters available on-demand without requiring them to be installed in the project's virtual environment.

Motivation

Reduce venv size: Development linters add significant overhead to the virtual environment
Version consistency: Pin linter versions to match .pre-commit-config.yaml for consistency across all workflows
Consistency: Same linter versions across all environments without requiring venv sync
Faster setup: Linters are cached globally and don't need reinstallation per project
Cleaner dependencies: Remove linter dependencies from [dependency-groups].dev in pyproject.toml

Proposed Changes

1. Add Makefile variables and `uv` installation task

Add variables at the top of the Makefile to locate uv and uvx:

# UV tool runner variables
UV := $(shell command -v uv 2>/dev/null || echo $(HOME)/.cargo/bin/uv)
UVX := $(UV) tool run

Create a new task that ensures uv is installed (quietly succeeds if already present):

.PHONY: uv
uv:                                 ## 📦  Install uv package manager
	@if command -v uv >/dev/null 2>&1; then \
		exit 0; \
	elif command -v brew >/dev/null 2>&1; then \
		echo "🍺  Installing uv via Homebrew..."; \
		brew install uv; \
	else \
		echo "📥  Installing uv via official install script..."; \
		curl -LsSf https://astral.sh/uv/install.sh | sh; \
	fi

Key design decisions:

Quietly succeeds (no output) if uv is already installed
Only shows output when actually installing uv
Uses Homebrew first (preferred method), falls back to install script
Sets $(UV) and $(UVX) variables for use throughout the Makefile

2. Make all linter targets depend on `uv` task

Every linter target should depend on the uv task to ensure uv is available:

autoflake: uv                       ## 🧹  Strip unused imports / vars
	@echo "🧹 autoflake $(TARGET)..."
	@$(UVX) [email protected] --in-place --remove-all-unused-imports \
		--remove-unused-variables -r $(TARGET)

black: uv                           ## 🎨  Reformat code with black
	@echo "🎨  black $(TARGET)..." && $(UVX) [email protected] -l 200 $(TARGET)

isort: uv                           ## 🔀  Sort imports
	@echo "🔀  isort $(TARGET)..." && $(UVX) [email protected] $(TARGET)

flake8: uv                          ## 🐍  flake8 checks
	@echo "🐍 flake8 $(TARGET)..." && $(UVX) [email protected] $(TARGET)

pylint: uv                          ## 🐛  pylint checks
	@echo "🐛 pylint $(TARGET)..." && $(UVX) [email protected] $(TARGET)

mypy: uv                            ## 🏷️  mypy type-checking
	@echo "🏷️ mypy $(TARGET)..." && $(UVX) [email protected] $(TARGET)

bandit: uv                          ## 🛡️  bandit security scan
	@echo "🛡️ bandit $(TARGET)..."
	@if [ -d "$(TARGET)" ]; then \
		$(UVX) [email protected] -r $(TARGET); \
	else \
		$(UVX) [email protected] $(TARGET); \
	fi

ruff: uv                            ## ⚡  Ruff lint + format
	@echo "⚡ ruff $(TARGET)..." && $(UVX) [email protected] check $(TARGET) && $(UVX) [email protected] format $(TARGET)

interrogate: uv                     ## 📝 Docstring coverage
	@echo "📝  interrogate - checking docstring coverage..."
	@$(UVX) [email protected] -vv mcpgateway || true

3. Convert Python linter targets to use `$(UVX)` with pinned versions

Update all Python-based linter targets to use $(UVX) with pinned versions matching .pre-commit-config.yaml.

IMPORTANT: Linter versions in uvx calls MUST match the versions specified in .pre-commit-config.yaml to ensure consistency between Makefile linting and pre-commit hooks.

Linters to convert with version pinning:

From .pre-commit-config.yaml:

[email protected] (from .pre-commit-config.yaml line 409)
[email protected] (from .pre-commit-config.yaml line 316)
[email protected] (from .pre-commit-config.yaml line 535)

From pyproject.toml (for linters not in pre-commit):

Note: For tools that require special handling (like prospector[with_everything]), use the --with syntax:

prospector: uv                      ## 🔬 Comprehensive code analysis
	@echo "🔬  prospector - running comprehensive analysis..."
	@$(UVX) --with prospector[with_everything]@1.17.3 prospector mcpgateway || true

4. Remove linters from `pyproject.toml`

Remove the following packages from [dependency-groups].dev:

# Remove these lines:
"autoflake>=2.3.1",
"bandit>=1.8.6",
"black>=25.9.0",
"check-manifest>=0.50",
"fawltydeps>=0.20.0",
"flake8>=7.3.0",
"importchecker>=3.0",
"interrogate>=1.7.0",
"isort>=6.1.0",
"mypy>=1.18.2",
"prospector[with_everything]>=1.17.3",
"pydocstyle>=6.3.0",
"pylint>=3.3.9",
"pylint-pydantic>=0.3.5",
"pyrefly>=0.35.0",
"pyright>=1.1.406",
"pyroma>=5.0",
"pyspelling>=2.11",
"pytype>=2024.10.11",
"radon>=6.0.1",
"ruff>=0.13.3",
"unimport>=1.3.0",
"vulture>=2.14",
"yamllint>=1.37.1",

Keep these in dev dependencies:

Testing tools: pytest, pytest-* plugins, hypothesis
Coverage: coverage, coverage-badge, pytest-cov
Build/packaging: twine, bump2version, cookiecutter
Pre-commit: pre-commit (it manages its own tool versions)
Other dev tools: redis, websockets, pexpect, etc.

5. Update `fawltydeps` ignore list

Update [tool.fawltydeps].ignore_unused in pyproject.toml to remove the linters we've removed from dependencies.

6. Update Documentation

Update the following documentation files to reflect the transition to uvx:

CLAUDE.md: Update the "Code Quality Pipeline" section to mention uvx usage
DEVELOPING.md: Update any references to linter installation/usage
README.md: Update development setup instructions if they mention linters
docs/docs/development/building.md: Update build/development workflow documentation
docs/docs/development/developer-onboarding.md: Update onboarding instructions for new developers
docs/docs/development/review.md: Update code review documentation if it mentions linting

Each documentation file should:

Note that linters are now executed via uvx with pinned versions
Explain that linter versions match .pre-commit-config.yaml for consistency
Remove any instructions about installing linters in the venv
Add a note about the make uv target for installing uv (optional, auto-installed on first linter use)

Implementation Steps

✅ Add $(UV) and $(UVX) variables to Makefile
✅ Add uv installation task to Makefile (quiet if already installed)
✅ Identify linter versions from .pre-commit-config.yaml and pyproject.toml
✅ Convert each Python linter target to:
- Depend on uv task
- Use $(UVX) with pinned versions
✅ Test each converted target individually
✅ Remove linter packages from [dependency-groups].dev
✅ Update fawltydeps ignore list
✅ Update all documentation files listed above
✅ Update CI/CD workflows if they reference these tools
✅ Test full quality pipeline: make autoflake isort black flake8 bandit interrogate pylint
✅ Verify pre-commit hooks still work correctly

Success Criteria

All linter targets work correctly with $(UVX)
All linter targets depend on uv task
uv task quietly succeeds if uv is already installed
$(UV) and $(UVX) variables are properly set
Linter versions are explicitly pinned and match .pre-commit-config.yaml where applicable
Virtual environment size is reduced
No regression in linting coverage or functionality
CI/CD pipelines continue to work
Documentation accurately reflects the new workflow
New developers can follow updated documentation successfully

Version Consistency Strategy

To maintain version consistency:

Primary source of truth: .pre-commit-config.yaml for linters used in pre-commit hooks
Secondary source: pyproject.toml for linters not in pre-commit
Makefile implementation: Pin to exact versions from primary/secondary sources using $(UVX)
Future updates: When updating linter versions, update in this order:
- Update .pre-commit-config.yaml first
- Update Makefile uvx calls to match
- Remove from pyproject.toml if not needed elsewhere

Notes

pre-commit should remain in dev dependencies as it manages its own hook versions
Some targets like interrogate and prospector currently install via pip inside the target - these should be converted to $(UVX) as well
Tools with optional extras (like prospector[with_everything]) may need special $(UVX) --with syntax
Consider adding a make lint-versions target to show all pinned linter versions for debugging
The markdownlint target uses npm and should remain unchanged
Shell linting tools (shellcheck, shfmt) use system packages and should remain unchanged
The uv task dependency ensures uv is installed before any linter runs
The $(UV) variable points to the uv binary location
The $(UVX) variable expands to $(UV) tool run for running tools

References

uv documentation
uvx tool runner
Project .pre-commit-config.yaml

Oct 20 '25 07:10 jonpspri

[chore] Transition linter execution from local venv to uvx-driven

Transition linter execution from local venv to uvx-driven

Summary

Motivation

Proposed Changes

1. Add Makefile variables and uv installation task

2. Make all linter targets depend on uv task

3. Convert Python linter targets to use $(UVX) with pinned versions

Linters to convert with version pinning:

4. Remove linters from pyproject.toml

5. Update fawltydeps ignore list

6. Update Documentation

Implementation Steps

Success Criteria

Version Consistency Strategy

Notes

References

1. Add Makefile variables and `uv` installation task

2. Make all linter targets depend on `uv` task

3. Convert Python linter targets to use `$(UVX)` with pinned versions

4. Remove linters from `pyproject.toml`

5. Update `fawltydeps` ignore list