Docs for result persistence location is out of date and confusing.
First check
- [X] I added a descriptive title to this issue.
- [X] I used GitHub search to find a similar request and didn't find it 😇
Describe the issue
Link to part of the docs that needs to be updated: https://github.com/PrefectHQ/prefect/blob/b3d98004a6748f07feb94a0c1899800804635969/docs/3.0rc/develop/results.mdx#result-storage-location
I have seen people be super confused that the block is defined in the decorator and also that with_options is used instead of the flow decorator.
Also S3 as a block will no longer be supported in favor of the s3 bucket block from the prefect-aws library.
new_flow = my_flow.with_options(result_storage=S3(bucket_path="my-bucket"))
^this is super confusing
from prefect import flow, task
from prefect.filesystems import LocalFileSystem, S3
@flow(persist_result=True)
def my_flow():
my_task() # This task will use the flow's result storage
@task(persist_result=True)
def my_task():
...
my_flow() # The flow has no result storage configured and no parent, the local file system will be used.
# Reconfigure the flow to use a different storage type
new_flow = my_flow.with_options(result_storage=S3(bucket_path="my-bucket"))
new_flow() # The flow and task within it will use S3 for result storage.
Describe the proposed change
Most users when they set up result persistence for the first time, they will be using a flow decorator.
First tell them to create the block, then have them reference the block from the flow decorator.
Use S3Bucket instead of S3 so we don't encourage them to adopt something that is not supported.
from prefect_aws.s3 import S3Bucket
@flow(persist_result=True, result_storage=S3Bucket.load("result-storage"))
def asyncio_gather_sub_deployments(
sim_failure: SimulatedFailure = SimulatedFailure(), sleep_time_subflows: int = 0
):
Additional context
No response
I'll take this up today or tomorrow 🫡
Thanks! Adding info on this setting
ENV PREFECT_DEFAULT_RESULT_STORAGE_BLOCK
from #10925 could also be helpful.
I get this question from customers all the time:
Can I set global result location and forgo defining it again and again in each flow decorator?
Awesome, will definitely cover that!