flyte icon indicating copy to clipboard operation
flyte copied to clipboard

[Feature] Datacatalog cache deletion

Open pvditt opened this issue 1 year ago • 2 comments

Tracking issue

https://github.com/flyteorg/flyte/issues/2867

Why are the changes needed?

Flyte doesn't support the deletion of cached task executions. Adding cache eviction/cleanup gives Flyte users more control over their data helping with things such as GDPR compliance + reducing their cache size.

Opted to support just single task executions as invalidating an entire workflow's cache can run into timeout issues going down the nesting of nodes.

What changes were proposed in this pull request?

  • add flyteadmin endpoint to support the deletion of single task executions
  • add catalog endpoint to support the deletion of single task executions

How was this patch tested?

  1. unit tests

  2. Run workflow that caches task executions

grpcurl -plaintext -d '{"task_execution_id": {"node_execution_id": {"execution_id": {"project": "flytesnacks", "domain": "development", "name": "fbcf25e4303104b4b81e"}, "node_id": "n0"}, "task_id": {"project": "flytesnacks", "domain": "development", "name": "basics.cache_test.basic_cache", "version": "98XuDRSQW4hVWKYQdQfd1Q=="}, "retry_attempt": 0}}' localhost:8089 flyteidl.service.CacheService.EvictTaskExecutionCache

Setup process

  • if want to verify from console, need to pull down: https://github.com/flyteorg/flyteconsole/pull/835 and then manually update flyteconsole/packages/flyteidl-types/node_modules/@flyteorg/flyteidl/gen/pb-js/flyteidl.d.ts
        enum CatalogCacheStatus {
            CACHE_DISABLED = 0,
            CACHE_MISS = 1,
            CACHE_HIT = 2,
            CACHE_POPULATED = 3,
            CACHE_LOOKUP_FAILURE = 4,
            CACHE_PUT_FAILURE = 5,
            CACHE_SKIPPED = 6,
            CACHE_EVICTED = 7
        }

(NOTE - a follow up PR will be to get that change merged into console since console doesn't support CACHE_EVICT CatalogCacheStatus)

  • can also verify by checking artifact_data and artifacts tables

Screenshots

Check all the applicable boxes

  • [x] I updated the documentation accordingly.
  • [x] All new and existing tests passed.
  • [x] All commits are signed-off.

Related PRs

Docs link

pvditt avatar Dec 30 '23 02:12 pvditt

Codecov Report

Attention: Patch coverage is 67.56098% with 133 lines in your changes are missing coverage. Please review.

Project coverage is 59.07%. Comparing base (e07084c) to head (9e322a8).

Files Patch % Lines
flyteadmin/pkg/manager/impl/cache_manager.go 68.18% 65 Missing and 5 partials :warning:
datacatalog/pkg/repositories/gormimpl/artifact.go 75.92% 9 Missing and 4 partials :warning:
flytestdlib/catalog/client.go 0.00% 11 Missing :warning:
flytestdlib/catalog/datacatalog/transformer.go 0.00% 10 Missing :warning:
flytestdlib/catalog/noop_catalog.go 0.00% 10 Missing :warning:
...atacatalog/pkg/manager/impl/reservation_manager.go 0.00% 5 Missing and 2 partials :warning:
datacatalog/pkg/manager/impl/artifact_manager.go 88.00% 5 Missing and 1 partial :warning:
flytestdlib/catalog/datacatalog/datacatalog.go 87.23% 4 Missing and 2 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4655      +/-   ##
==========================================
+ Coverage   59.00%   59.07%   +0.07%     
==========================================
  Files         645      647       +2     
  Lines       55578    55972     +394     
==========================================
+ Hits        32792    33065     +273     
- Misses      20194    20301     +107     
- Partials     2592     2606      +14     
Flag Coverage Δ
unittests 59.07% <67.56%> (+0.07%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Dec 30 '23 02:12 codecov[bot]

put up change for console: https://github.com/flyteorg/flyteconsole/pull/851

pvditt avatar Mar 12 '24 21:03 pvditt