[flyteagent] Add `agents` and `agentForTaskTypes` in helm chart

[mao3267]

Tracking issue

#3936

Why are the changes needed?

We aim to enhance the default configuration by adding more customization options for agent users. Specifically, we are introducing custom agent deployments in AgentDeployments and defining task types for specific agents in AgentForTaskTypes. These additions enable greater flexibility across various scenarios.

What changes were proposed in this pull request?

1. Add AgentDeployments and AgentForTaskTypes in default config
2. Test the correctness of the default configuration content and verify the task type-to-agent matching.

How was this patch tested?

1. Modify the existing test to check new added labels (AgentDeployments, AgentForTaskTypes)
2. Add new test to check matching between task types and agents.

Labels

added

Setup process

git clone https://github.com/flyteorg/flyte.git
gh pr checkout 6239

Screenshots

Check all the applicable boxes

• [ ] I updated the documentation accordingly.
• [ ] All new and existing tests passed.
• [ ] All commits are signed-off.

Related PRs

#6179

Docs link

None

Summary by Bito

This PR enhances Flyte agent configuration by introducing custom agent deployments and task type mapping in Helm charts. The implementation adds timeouts for agent operations, configures custom agent endpoints, and establishes task type to agent mappings. The changes improve flexibility and customization options while including comprehensive test coverage.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 5

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at [email protected].

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 36.86%. Comparing base (ee919f8) to head (a490251). :warning: Report is 196 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #6239   +/-   ##
=======================================
  Coverage   36.86%   36.86%           
=======================================
  Files        1318     1318           
  Lines      134767   134767           
=======================================
+ Hits        49684    49685    +1     
+ Misses      80753    80752    -1     
  Partials     4330     4330           

Flag	Coverage Δ
unittests-datacatalog	51.58% <ø> (ø)
unittests-flyteadmin	51.87% <ø> (ø)
unittests-flytecopilot	30.99% <ø> (ø)
unittests-flytectl	62.29% <ø> (ø)
unittests-flyteidl	7.22% <ø> (ø)
unittests-flyteplugins	54.03% <ø> (ø)
unittests-flytepropeller	42.78% <ø> (ø)
unittests-flytestdlib	55.35% <ø> (+0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Future-Outlier]

This is the updated configmap! before configmap:

      agent-service:
        defaultAgent:
          defaultTimeout: 10s
          endpoint: k8s://flyteagent.flyte:8000
          insecure: true
          timeouts:
            GetTask: 10s

after

      agent-service:
        agentForTaskTypes:
        - noop_task: custom_agent
        agents:
          custom_agent:
            endpoint: k8s://flyte_custom_agent.flyte:8000
            insecure: true
        defaultAgent:
          defaultTimeout: 10s
          endpoint: k8s://flyteagent.flyte:8000
          insecure: true
          timeouts:
            CreateTask: 10s
            DeleteTask: 10s
            ExecuteTaskSync: 10s
            GetTask: 10s
        supportedTaskTypes: []

[flyte-bot]

Code Review Agent Run #5f1d05

Actionable Suggestions - 2

• docker/sandbox-bundled/manifests/complete-agent.yaml - 1

  • Empty supported task types array · Line 500-500
• flyteplugins/go/tasks/plugins/webapi/agent/config_test.go - 1

  • Consider enabling mock assertion check · Line 141-142

Review Details

• Files reviewed - 9 · Commit Range: 59b8eab..a490251

  • charts/flyte-binary/values.yaml
  • charts/flyte-core/values.yaml
  • docker/sandbox-bundled/manifests/complete-agent.yaml
  • docker/sandbox-bundled/manifests/complete.yaml
  • docker/sandbox-bundled/manifests/dev.yaml
  • flyteplugins/go/tasks/plugins/webapi/agent/config.go
  • flyteplugins/go/tasks/plugins/webapi/agent/config_test.go
  • go.mod
  • go.sum
• Files skipped - 2

  • charts/flyte-binary/README.md - Reason: Filter setting
  • charts/flyte-core/README.md - Reason: Filter setting
• Tools

  • Golangci-lint (Linter) - ✖︎ Failed
  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful

---

AI Code Review powered by

[flyte-bot]

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
Feature Improvement - Enhanced Agent Configuration in Helm Charts	- values.yaml - Added timeouts and custom agent configuration - values.yaml - Added agent timeouts and custom agent settings - complete-agent.yaml - Updated agent configuration and security tokens - complete.yaml - Updated security tokens and checksums - dev.yaml - Updated security tokens and checksums
Testing - Agent Configuration Testing Updates	- config.go - Added default agent deployment configuration - config_test.go - Added tests for agent deployment and task type matching
Other Improvements - Dependency Management Updates	- go.mod - Removed unused dependencies - go.sum - Cleaned up and removed unused package dependencies