Downloader component
One additional component we need is a component that fetches the image_path from where it is stored in the cloud since in our set up we don't automatically expose the underlying files to our pipelines. Just wanted to raise this here since users could need such a component in enterprise settings.
Quick thought: can we reuse LinkContentFetcher?
@sjrl has background on this.
This will definitely require an investigation to see if it can be made to generalize.
I brought it up initially because deepset requires this since we store our files in a different location from the images that are used to serve our pipelines.
So the downloader component we use in deepset is DeepsetFileDownloader https://github.com/deepset-ai/deepset-cloud-custom-nodes/blob/main/deepset_cloud_custom_nodes/augmenters/deepset_file_downloader.py which is specific to our set up.
After discussion with @julian-risch about this an initial idea for completing this issue could be creating a S3FileDownloader component which would mimic the current one we are using in deepset. Then as needed we can support more downloader options (e.g. for different cloud providers) as is requested. This way we don't need to worry about trying to make a general downloader component.