botocore
botocore copied to clipboard
Improve documentation for multiprocessing/threading around the `boto3.client` method.
Currently, invoking boto3.client to create a client calls create_client on the default Boto3 session. This can lead to concurrency issues when done across parallelization primitives unexpectedly. We should update the documentation to clarify both in the client and session docs that calling this method inside threads may not be safe. The safe approach is to create a client in the parent thread/process either with the default session or a user instantiated session. This client can be shared as needed.
Hi, @nateprewitt I am interested in this and I will work on it. I have some questions:
- Is that should be opened in the boto3 repo instead? As the docs you mention are there and I will open a pull request in that repo not here. Should I open another issue in the boto3 and mention this issue or just make the pull request directly?
- Do you want to make it appear like a note, a normal text, or something else?
- Should I put what you have written in a little format in the note? or do you have another better description?
Hi @mustafaelghrib,
Is that should be opened in the boto3 repo instead?
That's correct that the changes needed here will be done in the boto3 repository. Specifically here in the guides section for client and session.
You can reference this issue in your boto3 PR, I wouldn't worry about creating a duplicate there.
Do you want to make it appear like a note, a normal text, or something else?
A small note or single sentence at the end of each section is likely sufficient.
Should I put what you have written in a little format in the note? or do you have another better description?
This would be a reasonable snippet for the note:
"Note that boto3.client uses a single, shared session for all calls. This can lead to concurrency issues unexpectedly when done across parallelization primitives. We recommend managing your own session(s) with concurrent code."
Okay @nateprewitt I opened a pull request, you could check it now