llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

feat(providers): implement ProviderContext for portable storage directory

Open rhuss opened this issue 6 months ago • 0 comments
trafficstars

[This PR implements #2139 but generalizes to separate runtime environments/contexts from specific provider configuration]

This PR introduces a new ProviderContext object that decouples execution environment concerns from provider configuration. In this first iteration, the ProviderContext is used to introduce storage directory which providers should use as base directory (e.g. for SQLite dbs). This allows removing absolute paths from templates and make the whole distribution more portable across environments. E.g. the current default to store everything in ~/.llama might be difficult to use in a containerized environment where the notion of a $HOME directory might differ from base image to base image.

Solution Details

  • Introduced a new ProviderContext class that encapsulates runtime environment information
  • Separates concerns between static configuration and runtime variables that depend on execution context - Updated provider interfaces to follow a consistent context, config, deps pattern
  • Provides consistent access to environment information across the codebase
  • Implemented a lookup for defining the storage directory:
    • CLI option (--storage-dir)
    • Environment variable (LLAMA_STACK_STORAGE_DIR)
    • Default distribution directory

[!NOTE]

  • Only inline (local) providers have been updated in this PR
  • Remote adapters and other provider types are left out for the moment

Next Steps

  • Add ProviderContext objects to remote adapters (for now it is added only for local providers)
  • Update templates to use relative pathes

rhuss avatar May 12 '25 10:05 rhuss