airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Published 20 hours ago • verified publisher icon apache

KubernetesPodOperator prints empty log lines without respecting the log format

Open karakanb opened this issue 1 year ago • 24 comments

Apache Airflow Provider(s)

cncf-kubernetes

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.14.0 apache-airflow-providers-celery==3.5.1 apache-airflow-providers-cncf-kubernetes==7.12.0 apache-airflow-providers-common-io==1.1.0 apache-airflow-providers-common-sql==1.10.0 apache-airflow-providers-discord==3.5.0 apache-airflow-providers-docker==3.8.2 apache-airflow-providers-elasticsearch==5.3.0 apache-airflow-providers-ftp==3.7.0 apache-airflow-providers-google==10.13.0 apache-airflow-providers-grpc==3.4.0 apache-airflow-providers-hashicorp==3.6.0 apache-airflow-providers-http==4.8.0 apache-airflow-providers-imap==3.5.0 apache-airflow-providers-microsoft-azure==8.4.0 apache-airflow-providers-mysql==5.5.0 apache-airflow-providers-odbc==4.2.0 apache-airflow-providers-openlineage==1.3.0 apache-airflow-providers-postgres==5.9.0 apache-airflow-providers-redis==3.5.0 apache-airflow-providers-sendgrid==3.4.0 apache-airflow-providers-sftp==4.8.0 apache-airflow-providers-slack==8.5.1 apache-airflow-providers-snowflake==5.2.1 apache-airflow-providers-sqlite==3.6.0 apache-airflow-providers-ssh==3.9.0

Apache Airflow version

2.8.0

Operating System

Debian 12 (bookworm)

Deployment

Official Apache Airflow Helm Chart

Deployment details

deployed on digitalocean k8s. I have a custom log format as '%%(asctime)s | %%(levelname)s | %%(message)s'

What happened

I have recently upgraded the cncf-kubernetes provider from 7.4.2 to 7.12.0 and started seeing weird log lines being printed out in-between application logs:

2024-01-03T21:31:59.286+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3
2024-01-03T21:32:00.313+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3
2024-01-03T21:32:00.694053856Z 
2024-01-03T21:32:00.694107912Z 
2024-01-03T21:32:00.694384093Z 
2024-01-03T21:32:00.694416033Z 
2024-01-03T21:32:00.694514264Z 
2024-01-03T21:32:00.694543915Z 
2024-01-03T21:32:00.694553314Z 
2024-01-03T21:32:00.694557782Z 
2024-01-03T21:32:00.694629859Z 
2024-01-03T21:32:00.694642682Z 
2024-01-03T21:32:01.348+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3

this also happens while the application is running:

2024-01-03T21:32:04.814+0000 | INFO | [base] application log line 1
2024-01-03T21:32:05.234+0000 | INFO | [base] application log line 2
2024-01-03T21:32:05.526911055Z 
2024-01-03T21:32:05.529+0000 | INFO | [base] application log line 3
2024-01-03T21:32:05.529+0000 | INFO | [base] application log line 4
2024-01-03T21:32:05.654767914Z 
2024-01-03T21:32:05.654830820Z 
2024-01-03T21:32:05.654857369Z

As you can see the log lines have a different format than the actual airflow log format, and they seem to contain no messages. This causes confusion, and for a few cases where I parse the logs it broke my existing code.

I have checked through external log collectors as well as via tailing the pod logs and I can confirm that the logs do not come from the application/pod side, and being logged somewhere on the airflow side.

What you think should happen instead

there should be no log lines printed that doesn't respect the logging format.

How to reproduce

Create a pod with KubernetesPodOperator, it happens to me consistently.

Anything else

Every time I run a pod it happens.

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

karakanb avatar Jan 03 '24 21:01 karakanb

This log is very strange; for example, for the pod starting wait, the check is running in a loop, and there are no other log statements: https://github.com/apache/airflow/blob/2c15dc9dd1b360bcb3b6d6c2e41e50ef0d97801e/airflow/providers/cncf/kubernetes/utils/pod_manager.py#L355-L366

These log messages come from something else, probably from the TaskContextLogger since you are using Airflow 2.8.0. Could you try to deactivate the feature and test the KPO? https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#enable-task-context-logger

hussein-awala avatar Jan 03 '24 22:01 hussein-awala

Agree log is very strange, @karakanb I am not able to replicate this with 2.8.0 & cncf-kubernetes==7.12.0, kindly do update us if you do not see this issue after disabling TaskContextLogger

[2024-01-04, 09:15:41 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:42 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:43 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:44 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:45 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:46 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:47 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:48 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:49 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:50 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:51 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:52 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:53 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:54 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:55 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:56 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:57 UTC] {pod_manager.py:359} WARNING - Pod not yet started: passing-test-o9zwtu7g [2024-01-04, 09:15:58 UTC] {pod_manager.py:460} INFO - [base] hello world [2024-01-04, 09:15:58 UTC] {pod_manager.py:488} WARNING - Pod passing-test-o9zwtu7g log read interrupted but container base still running [2024-01-04, 09:15:59 UTC] {pod_manager.py:460} INFO - [base] hello world [2024-01-04, 09:15:59 UTC] {pod.py:848} INFO - Deleting pod: passing-test-o9zwtu7g [2024-01-04, 09:15:59 UTC] {taskinstance.py:1138} INFO - Marking task as SUCCESS. dag_id=k8pod_operator, task_id=passing-task, execution_date=20240104T091538, start_date=20240104T091541, end_date=20240104T091559 [2024-01-04, 09:15:59 UTC] {local_task_job_runner.py:234} INFO - Task exited with return code 0 [2024-01-04, 09:15:59 UTC] {taskinstance.py:3281} INFO - 0 downstream tasks scheduled from follow-on schedule check

vatsrahul1001 avatar Jan 04 '24 10:01 vatsrahul1001

I can confirm that the issue is still there even after I disabled the context logger:

[logging]
colored_console_log = False
enable_task_context_logger = False
log_format = %%(asctime)s | %%(levelname)s | %%(message)s
remote_logging = False

karakanb avatar Jan 05 '24 15:01 karakanb

@karakanb, since you've upgraded the CNCF Kubernetes provider from version 7.4.2 to 7.12.0, does that imply you weren't experiencing the issue with 7.4.2 and only observed it after the upgrade to 7.12.0?

vatsrahul1001 avatar Jan 06 '24 04:01 vatsrahul1001

Exactly, the issue wasn't happening in 7.4.2, it has started happening in 7.12.0, although I have updated pretty much all the other providers as well as the airflow version at the same time. Let me test it by downgrading the cncf provider.

karakanb avatar Jan 06 '24 09:01 karakanb

The issue disappeared when I went back to 7.4.2, which means it has nothing to do with Airflow 2.8.0 or my environment as far as I can tell. I have upgraded to v7.13.0 and the issue persists there as well.

karakanb avatar Jan 06 '24 11:01 karakanb

cc: @dstandish - likely #35416 was not THAT inconsequential

potiuk avatar Jan 06 '24 17:01 potiuk

@potiuk according to the provided log:

2024-01-03T21:31:59.286+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3
2024-01-03T21:32:00.313+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3
2024-01-03T21:32:00.694053856Z 
2024-01-03T21:32:00.694107912Z 
2024-01-03T21:32:00.694384093Z 
2024-01-03T21:32:00.694416033Z 
2024-01-03T21:32:00.694514264Z 
2024-01-03T21:32:00.694543915Z 
2024-01-03T21:32:00.694553314Z 
2024-01-03T21:32:00.694557782Z 
2024-01-03T21:32:00.694629859Z 
2024-01-03T21:32:00.694642682Z 
2024-01-03T21:32:01.348+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3

the empty log messages are related to this loop: https://github.com/apache/airflow/blob/2c15dc9dd1b360bcb3b6d6c2e41e50ef0d97801e/airflow/providers/cncf/kubernetes/utils/pod_manager.py#L355-L366

and most probably logged by a 3rd party lib (tenacity.retry or kubernetes read_namespaced_pod methods) 🤔

hussein-awala avatar Jan 06 '24 17:01 hussein-awala

@potiuk I also tried reproducing this as mentioned here and not able to repro with cncf 7.12.0, I also suspects it is being logged by some 3rd party lib.

vatsrahul1001 avatar Jan 06 '24 17:01 vatsrahul1001

Yeah. It's just strange changing provider version fixes it - but yeah it could be caused also by some changes to provider dependencies between those two versions (but it does not look like it could be the reason):

git diff providers-cncf-kubernetes/7.4.2..providers-cncf-kubernetes/7.12.0 -- airflow/providers/cncf/kubernetes/provider.yaml

diff --git a/airflow/providers/cncf/kubernetes/provider.yaml b/airflow/providers/cncf/kubernetes/provider.yaml
index 11a661e63e..f0b35ca392 100644
--- a/airflow/providers/cncf/kubernetes/provider.yaml
+++ b/airflow/providers/cncf/kubernetes/provider.yaml
@@ -22,7 +22,17 @@ description: |
     `Kubernetes <https://kubernetes.io/>`__

 suspended: false
+source-date-epoch: 1703288121
 versions:
+  - 7.12.0
+  - 7.11.0
+  - 7.10.0
+  - 7.9.0
+  - 7.8.0
+  - 7.7.0
+  - 7.6.0
+  - 7.5.1
+  - 7.5.0
   - 7.4.2
   - 7.4.1
   - 7.4.0
@@ -65,7 +75,8 @@ versions:
   - 1.0.0

 dependencies:
-  - apache-airflow>=2.4.0
+  - aiofiles>=23.2.0
+  - apache-airflow>=2.6.0
   - asgiref>=3.5.2
   - cryptography>=2.0.0
   # The Kubernetes API is known to introduce problems when upgraded to a MAJOR version. Airflow Core
@@ -100,7 +111,6 @@ integrations:
 operators:
   - integration-name: Kubernetes
     python-modules:
-      - airflow.providers.cncf.kubernetes.operators.kubernetes_pod
       - airflow.providers.cncf.kubernetes.operators.pod
       - airflow.providers.cncf.kubernetes.operators.spark_kubernetes
       - airflow.providers.cncf.kubernetes.operators.resource
@@ -337,3 +347,6 @@ config:
         type: string
         example: ~
         default: ""
+
+executors:
+  - airflow.providers.cncf.kubernetes.kubernetes_executor.KubernetesExecutor

potiuk avatar Jan 06 '24 17:01 potiuk

Here's the dockerfile I use for this deployment:

FROM apache/airflow:2.8.0-python3.11

ENV PYTHONPATH "${PYTHONPATH}:${AIRFLOW_HOME}"
ENV SQLALCHEMY_SILENCE_UBER_WARNING=1

USER root
RUN apt-get update && apt-get install -y git gcc g++ unixodbc unixodbc-dev make nano curl

USER airflow

COPY requirements.txt /
RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" -r /requirements.txt

COPY . .

The requirements.txt:

apache-airflow-providers-amazon==8.14.0
apache-airflow-providers-celery==3.5.1
apache-airflow-providers-cncf-kubernetes==7.13.0
apache-airflow-providers-common-sql==1.10.0
apache-airflow-providers-discord==3.5.0
apache-airflow-providers-google==10.13.0
apache-airflow-providers-slack==8.5.1
apache-airflow-providers-snowflake==5.2.1
apache-airflow===2.8.0
deepdiff==6.7.1
isort==5.13.2
mypy==1.8.0
pendulum==2.1.2
pydantic==2.5.3
pytest-cov==4.1.0
pytest==7.4.3
pyyaml==6.0.1 
types-PyYAML==6.0.1
requests-mock==1.11.0
ruff==0.1.9
tabulate==0.9.0
hvac==2.0.0
The full set of dependencies, in case it helps: 
adal==1.2.7
adlfs==2023.10.0
aiobotocore==2.9.0
aiofiles==23.2.1
aiohttp==3.8.6
aioitertools==0.11.0
aiosignal==1.3.1
alembic==1.13.0
amqp==5.2.0
annotated-types==0.6.0
anyio==4.1.0
apache-airflow==2.8.0
apache-airflow-providers-amazon==8.14.0
apache-airflow-providers-celery==3.5.1
apache-airflow-providers-cncf-kubernetes==7.13.0
apache-airflow-providers-common-io==1.1.0
apache-airflow-providers-common-sql==1.10.0
apache-airflow-providers-discord==3.5.0
apache-airflow-providers-docker==3.8.2
apache-airflow-providers-elasticsearch==5.3.0
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-google==10.13.0
apache-airflow-providers-grpc==3.4.0
apache-airflow-providers-hashicorp==3.6.0
apache-airflow-providers-http==4.8.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==8.4.0
apache-airflow-providers-mysql==5.5.0
apache-airflow-providers-odbc==4.2.0
apache-airflow-providers-openlineage==1.3.0
apache-airflow-providers-postgres==5.9.0
apache-airflow-providers-redis==3.5.0
apache-airflow-providers-sendgrid==3.4.0
apache-airflow-providers-sftp==4.8.0
apache-airflow-providers-slack==8.5.1
apache-airflow-providers-snowflake==5.2.1
apache-airflow-providers-sqlite==3.6.0
apache-airflow-providers-ssh==3.9.0
apispec==6.3.0
argcomplete==3.2.1
asgiref==3.7.2
asn1crypto==1.5.1
async-timeout==4.0.3
attrs==23.1.0
Authlib==1.2.1
azure-batch==14.0.0
azure-common==1.1.28
azure-core==1.29.6
azure-cosmos==4.5.1
azure-datalake-store==0.0.53
azure-identity==1.15.0
azure-keyvault-secrets==4.7.0
azure-kusto-data==4.3.0
azure-mgmt-containerinstance==10.1.0
azure-mgmt-containerregistry==10.3.0
azure-mgmt-core==1.4.0
azure-mgmt-cosmosdb==9.3.0
azure-mgmt-datafactory==4.0.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==23.0.1
azure-mgmt-storage==21.1.0
azure-nspkg==3.0.2
azure-servicebus==7.11.4
azure-storage-blob==12.19.0
azure-storage-file-datalake==12.14.0
azure-storage-file-share==12.15.0
azure-synapse-artifacts==0.18.0
azure-synapse-spark==0.7.0
Babel==2.14.0
backoff==2.2.1
bcrypt==4.1.2
beautifulsoup4==4.12.2
billiard==4.2.0
blinker==1.7.0
boto3==1.33.13
botocore==1.33.13
cachelib==0.9.0
cachetools==5.3.2
cattrs==23.2.3
celery==5.3.6
certifi==2023.11.17
cffi==1.16.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.3.0
clickclick==20.10.2
colorama==0.4.6
colorlog==4.8.0
ConfigUpdater==3.2
connexion==2.14.2
coverage==7.4.0
cron-descriptor==1.4.0
croniter==2.0.1
cryptography==41.0.7
db-dtypes==1.2.0
decorator==5.1.1
deepdiff==6.7.1
Deprecated==1.2.14
dill==0.3.1.1
distlib==0.3.8
dnspython==2.4.2
docker==7.0.0
docutils==0.20.1
elastic-transport==8.11.0
elasticsearch==8.11.1
email-validator==1.3.1
eventlet==0.33.3
filelock==3.13.1
Flask==2.2.5
Flask-AppBuilder==4.3.10
Flask-Babel==2.0.0
Flask-Caching==2.1.0
Flask-JWT-Extended==4.6.0
Flask-Limiter==3.5.0
Flask-Login==0.6.3
Flask-Session==0.5.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==1.2.1
flower==2.0.1
frozenlist==1.4.1
fsspec==2023.12.2
gcloud-aio-auth==4.2.3
gcloud-aio-bigquery==7.0.0
gcloud-aio-storage==9.0.0
gcsfs==2023.12.2.post1
gevent==23.9.1
google-ads==22.1.0
google-api-core==2.15.0
google-api-python-client==2.111.0
google-auth==2.25.2
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.0
google-cloud-aiplatform==1.38.1
google-cloud-appengine-logging==1.4.0
google-cloud-audit-log==0.2.5
google-cloud-automl==2.12.0
google-cloud-batch==0.17.6
google-cloud-bigquery==3.14.1
google-cloud-bigquery-datatransfer==3.13.0
google-cloud-bigquery-storage==2.24.0
google-cloud-bigtable==2.22.0
google-cloud-build==3.22.0
google-cloud-compute==1.15.0
google-cloud-container==2.36.0
google-cloud-core==2.4.1
google-cloud-datacatalog==3.17.0
google-cloud-dataflow-client==0.8.6
google-cloud-dataform==0.5.5
google-cloud-dataplex==1.10.0
google-cloud-dataproc==5.8.0
google-cloud-dataproc-metastore==1.14.0
google-cloud-dlp==3.14.0
google-cloud-kms==2.20.0
google-cloud-language==2.12.0
google-cloud-logging==3.9.0
google-cloud-memcache==1.8.0
google-cloud-monitoring==2.18.0
google-cloud-orchestration-airflow==1.10.0
google-cloud-os-login==2.12.0
google-cloud-pubsub==2.19.0
google-cloud-redis==2.14.0
google-cloud-resource-manager==1.11.0
google-cloud-run==0.10.1
google-cloud-secret-manager==2.17.0
google-cloud-spanner==3.40.1
google-cloud-speech==2.23.0
google-cloud-storage==2.14.0
google-cloud-storage-transfer==1.10.0
google-cloud-tasks==2.15.0
google-cloud-texttospeech==2.15.0
google-cloud-translate==3.13.0
google-cloud-videointelligence==2.12.0
google-cloud-vision==3.5.0
google-cloud-workflows==1.13.0
google-crc32c==1.5.0
google-re2==1.1
google-resumable-media==2.7.0
googleapis-common-protos==1.62.0
graphviz==0.20.1
greenlet==3.0.2
grpc-google-iam-v1==0.13.0
grpcio==1.60.0
grpcio-gcp==0.2.2
grpcio-status==1.60.0
gunicorn==21.2.0
h11==0.14.0
httpcore==0.16.3
httplib2==0.22.0
httpx==0.23.3
humanize==4.9.0
hvac==2.0.0
idna==3.6
ijson==3.2.3
importlib-metadata==6.11.0
importlib-resources==6.1.1
inflection==0.5.1
iniconfig==2.0.0
isodate==0.6.1
isort==5.13.2
itsdangerous==2.1.2
Jinja2==3.1.2
jmespath==0.10.0
json-merge-patch==0.2
jsonpath-ng==1.6.0
jsonschema==4.20.0
jsonschema-specifications==2023.11.2
kombu==5.3.4
kubernetes==23.6.0
kubernetes-asyncio==24.2.3
lazy-object-proxy==1.10.0
ldap3==2.9.1
limits==3.7.0
linkify-it-py==2.0.2
lockfile==0.12.2
looker-sdk==23.20.0
lxml==4.9.3
Mako==1.3.0
Markdown==3.5.1
markdown-it-py==3.0.0
MarkupSafe==2.1.3
marshmallow==3.20.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mdit-py-plugins==0.4.0
mdurl==0.1.2
msal==1.26.0
msal-extensions==1.1.0
msrest==0.7.1
msrestazure==0.6.4
multidict==6.0.4
mypy==1.8.0
mypy-extensions==1.0.0
mysql-connector-python==8.0.29
mysqlclient==2.2.1
numpy==1.24.4
oauthlib==3.2.2
openlineage-integration-common==1.6.2
openlineage-python==1.6.2
openlineage_sql==1.6.2
opentelemetry-api==1.22.0
opentelemetry-exporter-otlp==1.22.0
opentelemetry-exporter-otlp-proto-common==1.22.0
opentelemetry-exporter-otlp-proto-grpc==1.22.0
opentelemetry-exporter-otlp-proto-http==1.22.0
opentelemetry-proto==1.22.0
opentelemetry-sdk==1.22.0
opentelemetry-semantic-conventions==0.43b0
ordered-set==4.1.0
packaging==23.2
pandas==2.1.4
pandas-gbq==0.20.0
paramiko==3.3.1
pathspec==0.12.1
pendulum==2.1.2
platformdirs==3.11.0
pluggy==1.3.0
ply==3.11
portalocker==2.8.2
prison==0.2.1
prometheus-client==0.19.0
prompt-toolkit==3.0.43
proto-plus==1.23.0
protobuf==4.25.1
psutil==5.9.6
psycopg2-binary==2.9.9
pyarrow==11.0.0
pyarrow-hotfix==0.6
pyasn1==0.5.1
pyasn1-modules==0.3.0
pycparser==2.21
pydantic==2.5.3
pydantic_core==2.14.6
pydata-google-auth==1.8.2
Pygments==2.17.2
PyJWT==2.8.0
PyNaCl==1.5.0
pyodbc==5.0.1
pyOpenSSL==23.3.0
pyparsing==3.1.1
pytest==7.4.3
pytest-cov==4.1.0
python-daemon==3.0.1
python-dateutil==2.8.2
python-dotenv==1.0.0
python-http-client==3.3.7
python-ldap==3.4.4
python-nvd3==0.15.0
python-slugify==8.0.1
pytz==2023.3.post1
pytzdata==2020.1
PyYAML==6.0.1
redis==4.6.0
redshift-connector==2.0.918
referencing==0.32.0
requests==2.31.0
requests-mock==1.11.0
requests-oauthlib==1.3.1
requests-toolbelt==1.0.0
rfc3339-validator==0.1.4
rfc3986==1.5.0
rich==13.7.0
rich-argparse==1.4.0
rpds-py==0.13.2
rsa==4.9
ruff==0.1.9
s3transfer==0.8.2
scramp==1.4.4
sendgrid==6.11.0
setproctitle==1.3.3
shapely==2.0.2
six==1.16.0
slack-sdk==3.26.1
sniffio==1.3.0
snowflake-connector-python==3.6.0
snowflake-sqlalchemy==1.5.1
sortedcontainers==2.4.0
soupsieve==2.5
SQLAlchemy==1.4.50
sqlalchemy-bigquery==1.9.0
SQLAlchemy-JSONField==1.0.2
sqlalchemy-redshift==0.8.14
sqlalchemy-spanner==1.6.2
SQLAlchemy-Utils==0.41.1
sqlparse==0.4.4
sshtunnel==0.4.0
starkbank-ecdsa==2.2.0
statsd==4.0.1
tabulate==0.9.0
tenacity==8.2.3
termcolor==2.4.0
text-unidecode==1.3
tomlkit==0.12.3
tornado==6.4
types-PyYAML==6.0.1
typing_extensions==4.9.0
tzdata==2023.3
uc-micro-py==1.0.2
unicodecsv==0.14.1
universal-pathlib==0.1.4
uritemplate==4.1.1
urllib3==2.0.6
vine==5.1.0
virtualenv==20.25.0
watchtower==3.0.1
wcwidth==0.2.12
websocket-client==1.7.0
Werkzeug==2.2.3
wrapt==1.16.0
WTForms==3.1.1
yarl==1.9.4
zipp==3.17.0
zope.event==5.0
zope.interface==6.1

karakanb avatar Jan 08 '24 08:01 karakanb

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Jan 23 '24 00:01 github-actions[bot]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Feb 07 '24 00:02 github-actions[bot]

Please keep this open.

karakanb avatar Feb 07 '24 21:02 karakanb

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Feb 22 '24 00:02 github-actions[bot]

ah come on now, please stay open.

karakanb avatar Feb 22 '24 08:02 karakanb

I added pinned label to keep it open @karakanb

potiuk avatar Feb 22 '24 11:02 potiuk

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Mar 08 '24 00:03 github-actions[bot]

keep it open please

karakanb avatar Mar 15 '24 13:03 karakanb

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Mar 30 '24 00:03 github-actions[bot]

This issue has been closed because it has not received response from the issue author.

github-actions[bot] avatar Apr 06 '24 00:04 github-actions[bot]

please reopen this

karakanb avatar Apr 07 '24 09:04 karakanb

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Apr 24 '24 00:04 github-actions[bot]

There's not much I can report unfortunately. I am running v2.9.0 and the issue is still there.

karakanb avatar Apr 25 '24 14:04 karakanb