feat: add pyproject.toml support to generate-recipe command

[millsks]

🎯 Summary

This PR adds native support for generating conda recipes from pyproject.toml files to the rattler-build generate-recipe command. This integrates the core functionality from pyrattler-recipe-autogen directly into rattler-build, eliminating the need for a separate tool.

🚀 Features Added

New Command

rattler-build generate-recipe pyproject <pyproject.toml> [--output <recipe.yaml>]

Core Functionality

• ✅ Full pyproject.toml parsing - Extract project metadata, dependencies, and build system requirements
• ✅ Dependency conversion - Convert Python package dependencies to conda format with proper version constraints
• ✅ Schema version support - Configurable schema version with YAML language server headers
• ✅ Conda overrides - Support for tool.conda.recipe.* configuration sections
• ✅ Entry points handling - Convert project.scripts to conda recipe entry points
• ✅ Build system integration - Add build system requirements to host dependencies

📝 Example Usage

Input pyproject.toml:

[project]
name = "my-package"
version = "1.0.0"
description = "A sample Python package"
dependencies = [
    "requests>=2.25.0",
    "click>=8.0.0"
]

[project.scripts]
my-tool = "my_package.cli:main"

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[tool.conda.recipe]
schema_version = 1

[tool.conda.recipe.about]
license = "MIT"
homepage = "https://github.com/example/my-package"

Generated recipe.yaml:

# yaml-language-server: $schema=https://raw.githubusercontent.com/prefix-dev/recipe-format/main/schema.json
schema_version: 1

context:
  name: my-package
  version: "1.0.0"

package:
  name: ${{ name }}
  version: ${{ version }}

source:
  path: .

build:
  script: python -m pip install . -vv --no-deps --no-build-isolation
  entry_points:
    - my-tool = my_package.cli:main

requirements:
  host:
    - python
    - pip
    - setuptools
    - wheel
  run:
    - python
    - requests >=2.25.0
    - click >=8.0.0

about:
  description: A sample Python package
  license: MIT
  homepage: https://github.com/example/my-package

tests:
  - python:
      imports:
        - my_package
      commands:
        - my-tool --help

🔧 Implementation Details

Files Added/Modified

• src/recipe_generator/pyproject.rs - Complete pyproject.toml parsing and recipe generation logic
• src/recipe_generator/serialize.rs - Added schema_version field support to Recipe struct
• src/recipe_generator/luarocks/mod.rs - Integrated pyproject command into CLI
• rust-tests/src/lib.rs - Added comprehensive integration tests

Key Components

1. Dependency Conversion: Converts Python package names and version constraints to conda format
2. Schema Support: Configurable schema version with proper YAML headers for VS Code integration
3. Conda Overrides: Full support for tool.conda.recipe.* sections to customize any part of the recipe
4. Build System Integration: Automatically adds build system requirements to host dependencies

🧪 Testing

Unit Tests (11 tests - all passing)

• test_convert_python_to_conda_dependency - Package name conversion logic
• test_format_python_constraint - Version constraint formatting
• test_build_package_section - Package metadata generation
• test_build_requirements_section - Dependencies and build requirements
• test_build_about_section - About section with conda overrides
• test_build_context_section - Context variables handling
• test_build_context_section_dynamic_version - Dynamic version resolution
• test_build_schema_version - Schema version configuration
• test_resolve_dynamic_version - Dynamic version extraction from files
• test_apply_package_name_mapping - Python to conda package mapping
• test_format_yaml_with_schema - YAML output with schema headers

Integration Tests (2 tests - all passing)

• test_generate_recipe_pyproject_basic - End-to-end basic functionality test
• test_generate_recipe_pyproject_with_conda_overrides - Advanced conda overrides test

🎁 Benefits

• Unified tooling - No need for separate pyrattler-recipe-autogen tool
• Better developer experience - Single command for all recipe generation needs
• Consistency - Unified behavior and output format across all recipe types
• Performance - Native Rust implementation for better performance
• Maintainability - Single codebase to maintain and update

🔗 Related Issues

Closes #1848 - Generate a recipe.yaml from pyproject.toml using generate-recipe subcommand

📋 Checklist

• [x] Implementation follows existing code patterns and conventions
• [x] Comprehensive unit test coverage (11 tests)
• [x] Integration test coverage (2 tests)
• [x] All tests passing
• [x] Documentation through examples and test cases
• [x] Follows Rust best practices and error handling
• [x] Schema version support for forward compatibility
• [x] Conda override support for advanced use cases

🚦 Testing Instructions

1. Create a test pyproject.toml file:

cat > test_pyproject.toml << 'EOF'
[project]
name = "test-package"
version = "1.0.0"
dependencies = ["requests>=2.25.0"]

[build-system]
requires = ["setuptools", "wheel"]
EOF

2. Generate a recipe:

rattler-build generate-recipe pyproject test_pyproject.toml

3. Verify the output contains proper conda dependencies and schema headers

📚 Additional Notes

• The implementation is designed to be extensible for future enhancements
• Full backward compatibility with existing recipe generation functionality
• Follows the same patterns as the existing luarocks recipe generator
• Ready for production use with comprehensive error handling

🚀 Quick Demo

Want to test this feature quickly? Here's a complete example:

# Create a sample pyproject.toml
cat > sample_pyproject.toml << 'EOF'
[project]
name = "demo-package"
version = "0.1.0"
description = "A demo package for testing pyproject recipe generation"
dependencies = [
    "click>=8.0.0",
    "requests>=2.25.0"
]

[project.scripts]
demo-tool = "demo_package.main:cli"

[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"

[tool.conda.recipe.about]
license = "MIT"
license_file = "LICENSE"
EOF

# Generate the conda recipe
rattler-build generate-recipe pyproject --input sample_pyproject.toml --output demo_recipe.yaml

# View the generated recipe
cat demo_recipe.yaml

This will generate a complete conda recipe with:

• Proper dependency conversion (click>=8.0.0 → click >=8.0.0)
• Build system requirements in host dependencies
• Entry points configuration
• Schema version and VS Code language server support
• Conda-specific metadata from tool.conda.recipe.* sections

[millsks]

@wolfv would you or anyone else on the team mind reviewing and providing feedback? I believe this feature may be useful for those behind corporate networks that need to generate a recipe for their python project that may not be published anywhere. This option will allow them to create the recipe with their local pyproject.toml.

Even if it doesn't get merged in it was a good learning experience.

Thanks!

[wolfv]

I like the idea, will have to check out the code :)

[wolfv]

Hey @millsks this is definitely functionality that we are interested in!

I am wondering - currently it looks like not a lot of code is shared with the PyPI generator. Do you think you could change that? Ideally we'd only have one version of these functions / mappings.

@zelosleone maybe you can also take a look?

[zelosleone]

Some problems that needs fixing:

• We don't need to load the full toml data after you map them to json. You can get the specific parts instead for building requirements, platform etc.
• I pointed some functions that can be replaced with pypi module we already have and this should be easy to do since you are with the first function in the module mapping the entire toml to json correctly (assuming)
• We don't need to hardcode noarch either, we can just parse and map it to json correctly and let it just be handled by the code.

[millsks]

Hi @wolfv and @zelosleone. Thanks for the feedback and suggestions. I'll start working on it this week.