llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

WIP: (feat) Add meta synthetic data kit as an inline provider

Open alinaryan opened this issue 5 months ago • 0 comments

What does this PR do?

Adds comprehensive test suite for the synthetic data kit provider implementation, including both unit and integration tests. This ensures the provider's functionality, configuration handling, and error cases are properly validated according to Llama Stack's testing guidelines.

Test Plan

  1. Unit Tests (tests/unit/providers/inline/synthetic_data_generation/test_synthetic_data_kit.py):

    pytest tests/unit/providers/inline/synthetic_data_generation/test_synthetic_data_kit.py -v
    

    Verifies:

    • Configuration initialization and validation
    • Environment variable handling via sample_run_config()
    • Basic synthetic data generation
    • Filtering functionality
    • Custom model specification
  2. Integration Tests (tests/integration/providers/inline/synthetic_data_generation/test_synthetic_data_kit_integration.py):

    # Start vLLM server on port 8000 first
    python -m vllm.entrypoints.api_server --model meta-llama/Llama-3.2-3B-Instruct --port 8000
    
    # Then run integration tests
    pytest tests/integration/providers/inline/synthetic_data_generation/test_synthetic_data_kit_integration.py -v
    

    Verifies:

    • End-to-end provider functionality with LlamaStackAsLibraryClient
    • Error handling for invalid inputs
    • Environment configuration integration
    • Response format and content validation

Prerequisites:

  • vLLM server running locally on port 8000
  • Access to meta-llama/Llama-3.2-3B-Instruct model
  • Python environment with test dependencies installed

alinaryan avatar May 29 '25 20:05 alinaryan