flyte
flyte copied to clipboard
[Core feature] Override `task secret_requests` using `with_overrides`
Tracking issue
https://github.com/flyteorg/flyte/issues/5085
Why are the changes needed?
with_overrides
method should enable overriding secret_requests
property. See Tracking issue for more information.
What changes were proposed in this pull request?
- Adding
override_security_context
properties to TaskNodeOverride proto andOverrideSecurityContext
toNodeSpec
. - Implement overrides when
TaskExecutionMetadata
is created.
How was this patch tested?
Setup process
I ran flyte on my dev machine and tested my code with this script:
Testing Script
from flytekit import task, Secret, workflow
import flytekit
import typing
@task(secret_requests=[Secret("group1", key="key1")])
def foo(a: str) -> str:
return a + flytekit.current_context().secrets.get("group2", "key2")
@task(secret_requests=[Secret("group2", key="key2")])
def foo2(a: str) -> str:
return a + flytekit.current_context().secrets.get("group2", "key2")
@workflow(
failure_policy=flytekit.WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
)
def my_wf() -> typing.Tuple[str, str, str, str, str, str, str, str]:
overrides1 = {
"secret_requests": [Secret("group2", key="key2"), Secret("group3", key="key3")],
}
overrides2 = {
"secret_requests": [Secret("group3", key="key3")],
}
overrides3 = {}
str1 = foo(a="foo-fail-")
str2 = foo2(a="foo2-success-")
str3 = foo(a="foo-fail-").with_overrides()
str4 = foo2(a="foo2-success-").with_overrides()
str5 = foo(a="override1-success-").with_overrides(**overrides1)
str6 = foo2(a="override2-fail-").with_overrides(**overrides2)
str7 = foo(a="override3-fail-").with_overrides(**overrides3)
str8 = foo2(a="override3-success-").with_overrides(**overrides3)
return str1, str2, str3, str4, str5, str6, str7, str8
- Create the testing secret with these commands:
kubectl create secret -n flytesnacks-development generic group1 --from-literal=key1=super-secret-value1 kubectl create secret -n flytesnacks-development generic group2 --from-literal=key2=super-secret-value2 kubectl create secret -n flytesnacks-development generic group3 --from-literal=key3=super-secret-value3
- Execute the script and the result should be like ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{red}Failed}$ => ${\color{green}Success}$
Screenshots
Check all the applicable boxes
- [ ] I updated the documentation accordingly.
- [ ] All new and existing tests passed.
- [x] All commits are signed-off.
Related PRs
https://github.com/flyteorg/flytekit/pull/2365
Docs link
Thank you for opening this pull request! 🙌
These tips will help get your PR across the finish line:
- Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
- Sign off your commits (Reference: DCO Guide).