dapr
Large file download/upload size limitation for Bindings
What content needs to be created or modified?
There should be documentation regarding how to handle large file operations (e.g., upload or download) using the Dapr Binding API. This is especially relevant for bindings implemented on top of blob storage providers (e.g., Azure Blob Storage, AWS S3, GCP Cloud Storage). Currently, there is little to no guidance on best practices for efficiently handling large payloads, chunking strategies, timeouts, retries, or memory management in such use cases.
Describe the solution you'd like
I would like to see a dedicated section or guide in the Dapr documentation that:
- Covers potential limitations (e.g., payload size, HTTP timeouts).
Where should the new material be placed?
A good location would be under the Bindings section of the docs. Alternatively, it could be added to the documentation of specific binding components like Azure Blob Storage or AWS S3.
The associated pull request from dapr/dapr, dapr/components-contrib, or other Dapr code repos
N/A at the moment. This is a documentation improvement suggestion based on observed gaps in current guidance.
Additional context
This issue arises when using Dapr to handle file uploads or downloads through binding components backed by blob storage. Without clear guidance, users may struggle with performance, reliability, or hitting undocumented limitations. Including this information would greatly help developers using Dapr in file-heavy workflows.
Thanks @WhitWaldo for pointing after raise this question : https://discord.com/channels/778680217417809931/901141509267685436/1357647096504909855
Hi @tomflenner , I’m new to contributing to Dapr and would like to take on this issue to add a guide for handling large file uploads/downloads with the Bindings API. I think it’d be a great fit for my first contribution. Is it okay if I work on this? Looking forward to any feedback or suggestions! Thanks!!!!!!
Hi @tomflenner , I’m new to contributing to Dapr and would like to take on this issue to add a guide for handling large file uploads/downloads with the Bindings API. I think it’d be a great fit for my first contribution. Is it okay if I work on this? Looking forward to any feedback or suggestions! Thanks!!!!!!
We covered this briefly in a Discord chat recently - the issue here is less to write a guide on how to shove large files through the bindings API and more to add a warning to each of the state bindings that large files aren't encouraged as that's not what it's presently built to do (e.g. lack of streaming support). Adding streaming support would be one approach to resolving this, but that's outside the scope of this particular issue.
@tomflenner - It is certainly still worth having clearer guidance on data size usage on the bindings. I would do two things;
- Yes cover best practices and really limitations using bindings with file in a new section in the overview
- Put any binding specific information into the respective component, as you suggest
Will look out for the PR.
@tomflenner - Would love to get a PR contribution here and then get you a digital doc contributor badge for v1.16? Would you be able to submit something?
Hey @msfussell, will try to at least submit a draft of this and we can iterate over to make it shine 🙏🏻