flyte icon indicating copy to clipboard operation
flyte copied to clipboard

[Core feature] Override `task secret_requests` using `with_overrides`

Open Mecoli1219 opened this issue 3 months ago • 2 comments

Tracking issue

https://github.com/flyteorg/flyte/issues/5085

Why are the changes needed?

with_overrides method should enable overriding secret_requests property. See Tracking issue for more information.

What changes were proposed in this pull request?

  • Adding override_security_context properties to TaskNodeOverride proto and OverrideSecurityContext to NodeSpec.
  • Implement overrides when TaskExecutionMetadata is created.

How was this patch tested?

Setup process

I ran flyte on my dev machine and tested my code with this script:

Testing Script
from flytekit import task, Secret, workflow
import flytekit
import typing


@task(secret_requests=[Secret("group1", key="key1")])
def foo(a: str) -> str:
    return a + flytekit.current_context().secrets.get("group2", "key2")


@task(secret_requests=[Secret("group2", key="key2")])
def foo2(a: str) -> str:
    return a + flytekit.current_context().secrets.get("group2", "key2")


@workflow(
    failure_policy=flytekit.WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE
)
def my_wf() -> typing.Tuple[str, str, str, str, str, str, str, str]:
    overrides1 = {
        "secret_requests": [Secret("group2", key="key2"), Secret("group3", key="key3")],
    }
    overrides2 = {
        "secret_requests": [Secret("group3", key="key3")],
    }
    overrides3 = {}
    str1 = foo(a="foo-fail-")
    str2 = foo2(a="foo2-success-")
    str3 = foo(a="foo-fail-").with_overrides()
    str4 = foo2(a="foo2-success-").with_overrides()
    str5 = foo(a="override1-success-").with_overrides(**overrides1)
    str6 = foo2(a="override2-fail-").with_overrides(**overrides2)
    str7 = foo(a="override3-fail-").with_overrides(**overrides3)
    str8 = foo2(a="override3-success-").with_overrides(**overrides3)

    return str1, str2, str3, str4, str5, str6, str7, str8
  1. Create the testing secret with these commands:
    kubectl create secret -n flytesnacks-development generic group1 --from-literal=key1=super-secret-value1
    kubectl create secret -n flytesnacks-development generic group2 --from-literal=key2=super-secret-value2
    kubectl create secret -n flytesnacks-development generic group3 --from-literal=key3=super-secret-value3
    
  2. Execute the script and the result should be like ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{green}Success}$ => ${\color{green}Success}$ => ${\color{red}Failed}$ => ${\color{red}Failed}$ => ${\color{green}Success}$

Screenshots

image

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [ ] All new and existing tests passed.
  • [x] All commits are signed-off.

Related PRs

https://github.com/flyteorg/flytekit/pull/2365

Docs link

Mecoli1219 avatar Apr 21 '24 09:04 Mecoli1219

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

welcome[bot] avatar Apr 21 '24 09:04 welcome[bot]