haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Add configurable environment variable names to S3FileDownloader init params

Open tstadel opened this issue 2 months ago • 4 comments

Is your feature request related to a problem? Please describe. Currently, the S3FileDownloader component uses hardcoded environment variable names for AWS configuration. This creates limitations when working with multiple S3 buckets or different AWS configurations within the same application.

Current limitation:

  • Users cannot easily download from multiple S3 buckets in the same application
  • Environment variable names are not configurable at the component level
  • Follows the pattern from initial custom component implementation but lacks flexibility for real-world use cases

Use Cases:

  • Multi-bucket applications: Download from different S3 buckets (e.g., PROD_BUCKET, STAGING_BUCKET)
  • Multi-environment setups: Use different AWS credentials for different services
  • Namespace isolation: Avoid environment variable conflicts in complex applications

Describe the solution you'd like Add optional initialization parameters to S3FileDownloader that allow users to specify custom environment variable names for AWS configuration.

Suggested API:

downloader = S3FileDownloader(
    ...,
    bucket_name_env="CUSTOM_S3_BUCKET_NAME"
)

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

tstadel avatar Sep 25 '25 08:09 tstadel