OpenMetadata icon indicating copy to clipboard operation
OpenMetadata copied to clipboard

Strange characters in query usage - Snowflake Ingestion Pipeline

Open triquinielas opened this issue 1 year ago • 0 comments

Hi,

I´m Carlos (Hotelbeds),

Affected module Ingestion

Describe the bug In the usage ingestion with snowflake source, we are having some problems in one database, the rest of the snowflake databases are working fine. When analysing the detail of the queries we have seen a common pattern, there are colons (:) in the queries that users use in the comments.

image

but,...the problem could be other simbol (chinesse simbol perhaps,..).

image

To Reproduce

If we filter like this: query_text not like '%:%', we prevent the usage ingest from failing: image

But this does not mean that the problem is in the colon symbol, but that filtering by this has prevented it from retrieving the query where the problem really is with another symbol. log query usage.txt

Expected behavior That the error in the ingestion of this type of error is controlled, as users can put any symbol in the comments.

Version:

  • OS: [e.g. iOS]
  • Python version:
  • OpenMetadata version: 1.2.2 openmetadata-managed-apis==1.2.2 openmetadata-ingestion[snowflake]==1.2.2 openmetadata-ingestion[airflow]==1.2.2 openmetadata-ingestion[tableau]==1.2.2 openmetadata-ingestion[postgres]==1.2.2

thanks in advance

triquinielas avatar Feb 09 '24 09:02 triquinielas