python
python copied to clipboard
kubernetes-client
Add support for bookmark events
What is the feature and why do you need it:
In kubernetes-client/python-base#234 the parsing of BOOKMARK events has been bypassed (and still is as of today). But those events exist for a reason and can be useful in some circumstances. A motivational example is this Airflow issue: https://github.com/apache/airflow/issues/21087.
As Airflows scheduler can be stopped and restarted, while many events occur on k8s, it would need to process BOOKMARK to recover its state. Sadly because those are ignored by the Python client, it cannot. Which manifests itself in varied ways (see the different issues referenced in the ticket above)
Describe the solution you'd like to see:
We'd like for the bookmark event to be parsed and yield by the watcher, so it is possible for clients to consume them.
(copied from kubernetes-client/python-base#286)
@roycaihw: This request has been marked as needing help from a contributor.
Guidelines
Please ensure that the issue body includes answers to the following questions:
- Why are we solving this issue?
- To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
- Does this issue have zero to low barrier of entry?
- How can the assignee reach out to you for help?
For more details on the requirements of such an issue, please see here and ensure that they are met.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.
In response to this:
Agreed this is nice to have.
/help
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@cansjt thanks for reporting this. i would like to make few comments:
What is the feature and why do you need it:
In kubernetes-client/python-base#234 the parsing of BOOKMARK events has been bypassed (and still is as of today). But those events exist for a reason and can be useful in some circumstances. A motivational example is this Airflow issue: apache/airflow#21087.
The parsing of the BOOKMARK event is not actually by passed. we just dont de-serialize the BOOKMARK event to the return_type eg: 'V1POD' etc. the bookmark event as a dict is still returned by the watcher to the client.
de serializing the event to the V1POD type etc is not easy because BOOKMARK event does not have all the required fields to do so. eg in case of V1POD, the container is None in BOOKMARK but required to be non empty for V1POD spec.
since all that really matter in the BOOKMARK event in the resoureVersion, we should be good with a dict object.
the problem that we have currently is the watcher does not update the resource_version to the one in the BOOKMARK event. which the #1796 should fix. once this fix is done, the 410 error won't happen for long duration watch.
As Airflows scheduler can be stopped and restarted, while many events occur on k8s, it would need to process BOOKMARK to recover its state. Sadly because those are ignored by the Python client, it cannot. Which manifests itself in varied ways (see the different issues referenced in the ticket above)
The BOOKMARK event is not send by the api server by default. it needs to be requested by the client by setting the allow_watch_bookmarks in the api call. by default it is false. in Airflow kubernetes executor it is not set to True.
so to take advantage of BOOKMARKS, in airflow we need to make the request with allow_watch_bookmark=true
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale