PrefectHQ
Document that args with leading underscores are not deployed
First check
- [X] I added a descriptive title to this issue.
- [X] I used the GitHub search to find a similar request and didn't find it.
- [X] I searched the Prefect documentation for this feature.
Prefect Version
2.x
Describe the current behavior
I was looking for ways to expose certain args only for testing and not for deployments. I though it might be a good idea to do this with a leading underscore.
@flow
def foo(a: int, _not_deployed="prod_path"):
pass
# in a test file
def test_foo():
with prefect_test_harness():
foo.should_validate_parameters = False
foo(1, _not_deployed="bar")
def test_foo_other_option():
with prefect_test_harness():
@flow
def test_foo():
foo.fn(1, _not_deployed="bar")
test_foo()
While trying to implement this i found that this is already the case - presumably because pydantic strips fields it considers private from the schema. This is great
Describe the proposed behavior
I have two points to of complaint.
No documentation for this behavior (that i could find)
If the parameter with a leading underscore has no default value the flow is still deployed - but the run will always fail because the args with a leading underscore will not be passed (and cannot be set).
TypeError: foo() missing 1 required positional argument: '_not_deployed'
Example Use
No response
Additional context
No response
This is unintentional but a fun find :) I wonder if we should keep this behavior, it seems convenient. If we want to do so, we'll need to error at definition time if it does not have a default value.
@madkinsz there currently certainly is some quirkiness to it. Enforcing a default would be great!
The ability to inject some parameters for unittests into a flow without having to expose them in the deployment should be a rather common usecase.
Without this the next best option is to use blocks - which can be defined in the prod environment - and saved for test after the harness has been created. But this certainly feels needlessly complicated, especially for cases where you just want to pass a string.
With pydantic>=2.0 a clear and informative error is now raised preventing leading underscores from parameter keyword arguments. I'm going to close this issue as resolved for now - if it becomes a desirable feature to work around Pydantic in these cases we can consider that in a separate enhancement request issue.
