docker-airflow icon indicating copy to clipboard operation
docker-airflow copied to clipboard

Published 20 hours ago • verified publisher icon puckel

KubernetesPodOperator Task FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'

Open kajari1verma opened this issue 6 years ago • 7 comments
trafficstars

Unable to load the config, contains a configuration error. Traceback (most recent call last): File "/usr/local/lib/python3.6/logging/config.py", line 565, in configure handler = self.configure_handler(handlers[name]) File "/usr/local/lib/python3.6/logging/config.py", line 738, in configure_handler result = factory(**kwargs) File "/usr/local/lib/python3.6/site-packages/airflow/utils/log/file_processor_handler.py", line 51, in init os.makedirs(self._get_log_directory()) File "/usr/local/lib/python3.6/os.py", line 210, in makedirs makedirs(head, mode, exist_ok) File "/usr/local/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/usr/local/airflow/logs/scheduler' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/airflow", line 21, in from airflow import configuration File "/usr/local/lib/python3.6/site-packages/airflow/init.py", line 36, in from airflow import settings, configuration as conf File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 262, in logging_class_path = configure_logging() File "/usr/local/lib/python3.6/site-packages/airflow/logging_config.py", line 72, in configure_logging raise e File "/usr/local/lib/python3.6/site-packages/airflow/logging_config.py", line 67, in configure_logging dictConfig(logging_config) File "/usr/local/lib/python3.6/logging/config.py", line 802, in dictConfig dictConfigClass(config).configure() File "/usr/local/lib/python3.6/logging/config.py", line 573, in configure '%r: %s' % (name, e)) ValueError: Unable to configure handler 'processor': [Errno 13] Permission denied: '/usr/local/airflow/logs/scheduler'

kajari1verma avatar Apr 09 '19 07:04 kajari1verma

When adding USER root then airflow webserver is started successfully but i am getting an error when executing a kubernetesPodOperator Task FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'

kajari1verma avatar Apr 10 '19 10:04 kajari1verma

same problem here: [2019-05-02 10:54:54,801] {base_task_runner.py:101} INFO - Job 2875: Subtask passing-task [2019-05-02 10:54:54,794] {__init__.py:305} INFO - Filling up the DagBag from /usr/local/airflow/dags/airflow-dags/podoperator.py [2019-05-02 10:54:56,795] {base_task_runner.py:101} INFO - Job 2875: Subtask passing-task [2019-05-02 10:54:56,794] {cli.py:517} INFO - Running <TaskInstance: kubenetes_sample.passing-task 2019-04-03T13:00:00+00:00 [running]> on host kubenetessamplepassingtask-3f84595a373540aa8222bbfdc01c8395 [2019-05-02 10:54:56,878] {__init__.py:1580} ERROR - [Errno 2] No such file or directory: '/root/.kube/config' Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 1441, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py", line 98, in execute config_file=self.config_file) File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 56, in get_kube_client return _load_kube_config(in_cluster, cluster_context, config_file) File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 38, in _load_kube_config config.load_kube_config(config_file=config_file, context=cluster_context) File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config config_persister=config_persister) File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file with open(filename) as f: FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config' [2019-05-02 10:54:56,888] {__init__.py:1603} INFO - Marking task as UP_FOR_RETRY

kohonen avatar May 02 '19 11:05 kohonen

Same issue

[2019-06-25 07:04:13,062] {__init__.py:1139} INFO - Dependencies all met for <TaskInstance: kubernetes_sample.passing-task 2019-06-25T07:04:05.886371+00:00 [queued]>
[2019-06-25 07:04:13,062] {__init__.py:1353} INFO -
--------------------------------------------------------------------------------
[2019-06-25 07:04:13,062] {__init__.py:1354} INFO - Starting attempt 1 of 1
[2019-06-25 07:04:13,062] {__init__.py:1355} INFO -
--------------------------------------------------------------------------------
[2019-06-25 07:04:13,069] {__init__.py:1374} INFO - Executing <Task(KubernetesPodOperator): passing-task> on 2019-06-25T07:04:05.886371+00:00
[2019-06-25 07:04:13,069] {base_task_runner.py:119} INFO - Running: ['airflow', 'run', 'kubernetes_sample', 'passing-task', '2019-06-25T07:04:05.886371+00:00', '--job_id', '6', '--raw', '-sd', 'DAGS_FOLDER/kubernetes_test.py', '--cfg_path', '/tmp/tmp5gqbbjmt']
[2019-06-25 07:04:13,550] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task /usr/local/lib/python3.6/site-packages/airflow/configuration.py:569: DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and airflow_home in the config file is deprecated. Please use only the AIRFLOW_HOME environment variable and remove the config file entry.
[2019-06-25 07:04:13,551] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   warnings.warn(msg, category=DeprecationWarning)
[2019-06-25 07:04:13,687] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:13,686] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=3734
[2019-06-25 07:04:13,817] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:13,817] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-06-25 07:04:14,088] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:14,087] {__init__.py:305} INFO - Filling up the DagBag from /usr/local/airflow/dags/git-sync/dags/kubernetes_test.py
[2019-06-25 07:04:14,403] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:14,403] {cli.py:517} INFO - Running <TaskInstance: kubernetes_sample.passing-task 2019-06-25T07:04:05.886371+00:00 [running]> on host dhruvs-dummy-pod
[2019-06-25 07:04:14,452] {__init__.py:1580} ERROR - [Errno 2] No such file or directory: '/root/.kube/config'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py", line 98, in execute
    config_file=self.config_file)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 56, in get_kube_client
    return _load_kube_config(in_cluster, cluster_context, config_file)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 38, in _load_kube_config
    config.load_kube_config(config_file=config_file, context=cluster_context)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config
    config_persister=config_persister)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'
[2019-06-25 07:04:14,453] {__init__.py:1611} INFO - Marking task as FAILED.
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task Traceback (most recent call last):
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/bin/airflow", line 32, in <module>
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     args.func(args)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return f(*args, **kwargs)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 523, in run
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     _run(args, dag, ti)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 442, in _run
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     pool=args.pool,
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 73, in wrapper
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return func(*args, **kwargs)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     result = task_copy.execute(context=context)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py", line 98, in execute
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config_file=self.config_file)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 56, in get_kube_client
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return _load_kube_config(in_cluster, cluster_context, config_file)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 38, in _load_kube_config
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config.load_kube_config(config_file=config_file, context=cluster_context)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config_persister=config_persister)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     with open(filename) as f:
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'
[2019-06-25 07:04:18,032] {logging_mixin.py:95} INFO - [2019-06-25 07:04:18,031] {jobs.py:2562} INFO - Task exited with return code 1```

null-sleep avatar Jun 25 '19 07:06 null-sleep

Just build new image with --build-arg AIRFLOW_DEPS="kubernetes"

MahaGamal avatar Jul 17 '19 12:07 MahaGamal

Just build new image with --build-arg AIRFLOW_DEPS="kubernetes"

This did not help solve the issue

bicepjai avatar Aug 06 '20 01:08 bicepjai

Same issue

[2019-06-25 07:04:13,062] {__init__.py:1139} INFO - Dependencies all met for <TaskInstance: kubernetes_sample.passing-task 2019-06-25T07:04:05.886371+00:00 [queued]>
[2019-06-25 07:04:13,062] {__init__.py:1353} INFO -
--------------------------------------------------------------------------------
[2019-06-25 07:04:13,062] {__init__.py:1354} INFO - Starting attempt 1 of 1
[2019-06-25 07:04:13,062] {__init__.py:1355} INFO -
--------------------------------------------------------------------------------
[2019-06-25 07:04:13,069] {__init__.py:1374} INFO - Executing <Task(KubernetesPodOperator): passing-task> on 2019-06-25T07:04:05.886371+00:00
[2019-06-25 07:04:13,069] {base_task_runner.py:119} INFO - Running: ['airflow', 'run', 'kubernetes_sample', 'passing-task', '2019-06-25T07:04:05.886371+00:00', '--job_id', '6', '--raw', '-sd', 'DAGS_FOLDER/kubernetes_test.py', '--cfg_path', '/tmp/tmp5gqbbjmt']
[2019-06-25 07:04:13,550] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task /usr/local/lib/python3.6/site-packages/airflow/configuration.py:569: DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and airflow_home in the config file is deprecated. Please use only the AIRFLOW_HOME environment variable and remove the config file entry.
[2019-06-25 07:04:13,551] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   warnings.warn(msg, category=DeprecationWarning)
[2019-06-25 07:04:13,687] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:13,686] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=3734
[2019-06-25 07:04:13,817] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:13,817] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-06-25 07:04:14,088] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:14,087] {__init__.py:305} INFO - Filling up the DagBag from /usr/local/airflow/dags/git-sync/dags/kubernetes_test.py
[2019-06-25 07:04:14,403] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task [2019-06-25 07:04:14,403] {cli.py:517} INFO - Running <TaskInstance: kubernetes_sample.passing-task 2019-06-25T07:04:05.886371+00:00 [running]> on host dhruvs-dummy-pod
[2019-06-25 07:04:14,452] {__init__.py:1580} ERROR - [Errno 2] No such file or directory: '/root/.kube/config'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py", line 98, in execute
    config_file=self.config_file)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 56, in get_kube_client
    return _load_kube_config(in_cluster, cluster_context, config_file)
  File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 38, in _load_kube_config
    config.load_kube_config(config_file=config_file, context=cluster_context)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config
    config_persister=config_persister)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'
[2019-06-25 07:04:14,453] {__init__.py:1611} INFO - Marking task as FAILED.
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task Traceback (most recent call last):
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/bin/airflow", line 32, in <module>
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     args.func(args)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return f(*args, **kwargs)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 523, in run
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     _run(args, dag, ti)
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 442, in _run
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     pool=args.pool,
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 73, in wrapper
[2019-06-25 07:04:14,478] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return func(*args, **kwargs)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     result = task_copy.execute(context=context)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py", line 98, in execute
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config_file=self.config_file)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 56, in get_kube_client
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     return _load_kube_config(in_cluster, cluster_context, config_file)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kube_client.py", line 38, in _load_kube_config
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config.load_kube_config(config_file=config_file, context=cluster_context)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     config_persister=config_persister)
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task   File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task     with open(filename) as f:
[2019-06-25 07:04:14,479] {base_task_runner.py:101} INFO - Job 6: Subtask passing-task FileNotFoundError: [Errno 2] No such file or directory: '/root/.kube/config'
[2019-06-25 07:04:18,032] {logging_mixin.py:95} INFO - [2019-06-25 07:04:18,031] {jobs.py:2562} INFO - Task exited with return code 1```

Hava you solved the problem? I encountered the same problem.

NightFaint avatar Aug 13 '20 11:08 NightFaint 

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1138, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1341, in _execute_task
    result = task_copy.execute(context=context)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 331, in execute
    config_file=self.config_file,
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/kube_client.py", line 126, in get_kube_client
    client_conf = _get_kube_config(in_cluster, cluster_context, config_file)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/kube_client.py", line 46, in _get_kube_config
    load_kube_config(client_configuration=cfg, config_file=config_file, context=cluster_context)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/refresh_config.py", line 123, in load_kube_config
    loader = _get_kube_config_loader_for_yaml_file(config_file, active_context=context, config_persister=None)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/refresh_config.py", line 105, in _get_kube_config_loader_for_yaml_file
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/airflow/.kube/config'

Is anyone has an update with this error

phamquang92 avatar Jul 22 '22 16:07 phamquang92