prefect icon indicating copy to clipboard operation
prefect copied to clipboard

Prefect blocks are not supported as flow run parameters

Open davidschrooten opened this issue 2 years ago • 5 comments

First check

  • [X] I added a descriptive title to this issue.
  • [X] I used the GitHub search to find a similar issue and didn't find it.
  • [X] I refreshed the page and this issue still occurred.
  • [X] I checked if this issue was specific to the browser I was using by testing with a different browser.

Bug summary

I am unable to select a block parameter from the Prefect Cloud UI when creating a flow run. In the JSON output when a block is selected I only get a JSON structure such as:

"$ref": {
      "block_document_id": "f14ba9fe-1f1f-43d2-9afa-8c0edf40ec02"
}

A similar issue was discussed on slack:

https://linen.prefect.io/t/14150855/i-don-t-understand-what-s-the-good-practice-to-work-with-blo

Yet the referenced PR didn't seem to have solved the issue.

Reproduction

Define a flow e.g.:

@flow(name="Sync MongoDB", task_runner=SequentialTaskRunner())
def sync_mongodb(
    credentials: MongoDBCredentials,
    database: Database = Database.develop,
    only_tables: Optional[List[Tables]] = None,
):
    main_collections(database, credentials, only_tables)
    extras_collections(database, credentials, only_tables)
    join_collections(database, credentials, only_tables)

Then try to login in the Prefect UI. When viewing the parameters as JSON you will get a block_document_id.

Error

Discussed above

Browers

  • [X] Chrome
  • [X] Firefox
  • [X] Safari
  • [X] Edge

Prefect version

Latest on cloud

Additional context

No response

davidschrooten avatar Oct 31 '23 15:10 davidschrooten

This seems like a important bug to me that's in prefect already for a couple of months. Not being able to load parameters with a Block type greatly degrades the UI experience. Now I have to use string parameters and load the blocks manually as a work around.

davidschrooten avatar Nov 01 '23 15:11 davidschrooten

Possibly connected to https://github.com/PrefectHQ/prefect/issues/10613

zhen0 avatar Nov 03 '23 21:11 zhen0

@davidschrooten, if you could provide an MRE for this that would be very useful

serinamarie avatar Nov 04 '23 14:11 serinamarie

I edited the title slightly to stop me thinking that all blocks are broken!

zhen0 avatar Nov 30 '23 22:11 zhen0

It’s surprising that this has been open for almost two years. I want to highlight some UX issues and broken workflows caused by block references not being treated as valid parameter values.

  1. Web client: parameters can’t be validated when using blocks

When using blocks as parameters, I have to disable parameter verification every single time. That creates the risk that parameters which should be validated are no longer validated. (Button i mean:)

Image
  1. Python SDK: impossible to pass block references cleanly

Passing block references to a new deployment isn’t possible (especially for secrets), which breaks otherwise straightforward workflows.

Example: • deploymentA runs on machineA and uploads files to an S3 bucket. • I want deploymentB on machineB to download from the same S3 bucket. • Intuitively, you’d expect to call run_deployment and pass the S3Bucket block. • This fails because deserializing SecretStr gives ********.

The next logical step – passing a block reference like {"$ref": "<block_document_id>"} – also fails, because run_deployment forces enforce_parameter_schema=True, and schema enforcement does not resolve block references.

This leaves only one workaround: pass the block name or block document ID as a plain string and manually load the block inside the second deployment. It works, but it’s clunky — and using deploymentB from the UI becomes a hassle because you now have to manually copy the block name or document ID. All of this feels unnecessary given that the underlying mechanisms already exist.

TL;DR: Passing blocks between flow runs is unnecessarily complicated, both in the UI and the SDK. A few small changes would remove a lot of friction and make these workflows much more natural.

streit-julian avatar Nov 22 '25 13:11 streit-julian