python Discoverer cache file is not thread safe

What happened (please include outputs or screenshots):

There are errors when using DynamicClient in multithreading env.

Sample error log like:

ERROR:root:load cache error: Expecting value: line 1 column 1 (char 0)

What you expected to happen:

No error, cache file write should be thread safe.

How to reproduce it (as minimally and precisely as possible):

Here is a sample script to reproduce, it's not same as what I'm using, but it will create a lot of concurrent writes.

import logging

from kubernetes import config, dynamic
from kubernetes.client import api_client
from concurrent.futures import ThreadPoolExecutor


def main():
    logging.basicConfig(level=logging.DEBUG)

    def get_api_client(_):
        client = dynamic.DynamicClient(
            api_client.ApiClient(configuration=config.load_kube_config())
        )
        client.resources.parse_api_groups(request_resources=False, update=True)
        return client

    logging.debug("start")

    with ThreadPoolExecutor(max_workers=10) as pool:
        list(pool.map(get_api_client, range(100)))

    logging.debug("done")


if __name__ == "__main__":
    main()

Anything else we need to know?:

Environment:

Kubernetes version (kubectl version): v1.25.7+eab9cc9
OS (e.g., MacOS 10.13.6): MacOS 13.2.1
Python version (python --version): Python 3.9.16
Python client version (pip list | grep kubernetes): 24.2.0

Idea to fix:

Add a threading.Lock to discovery.py

Guard _write_cache with lock so only one thread can write to it.

If the way to fix is agreed, I can prepare a pr for it.

Mar 30 '23 12:03 hemslo

/assign @fabianvf

Apr 10 '23 16:04 roycaihw

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jul 09 '23 17:07 k8s-triage-robot

/remove-lifecycle stale

Jul 10 '23 11:07 hemslo

Any idea on this? Do we support multithreading usage?

Jul 10 '23 11:07 hemslo

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jan 24 '24 03:01 k8s-triage-robot

/remove-lifecycle stale

Jan 24 '24 22:01 hemslo

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Apr 23 '24 23:04 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

May 24 '24 00:05 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Jun 23 '24 00:06 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Jun 23 '24 00:06 k8s-ci-robot