Error in Profiler Ingestions when upgrading from 1.4.3 to 1.5.1

[tanigawa-haruka]

Affected module Ingestion Framework

Describe the bug Upgrading from 1.4.3 to 1.5.1, I get the following error even with Profiler Ingestions without filter conditions. “databaseFilterPattern returned 0 result. At least 1 database must be returned by the filter pattern”.

To Reproduce Upgrade from 1.4.3 to 1.5.1

Expected behavior Profiler Ingestions run successfully

Version:

• OS: Linux
• OpenMetadata version: 1.5.1
• docker version: 20.10.25

Additional context slack queries https://openmetadata.slack.com/archives/C02B6955S4S/p1725960392869339

error log snowflake_profiler_YCPBp2Md-profiler.log

[rchlz]

I got the same error version：1.5.5

[TeddyCr]

@rchlz @tanigawa-haruka can you share more information regarding how you are running the profiler for what service and any ingestion filter that might be applied.

Going through 1.4.3 -> 1.5.1 upgrade path with a MySQL service running from the UI I could not reproduce it. Additional information may help us here.

thanks

[tanigawa-haruka]

@TeddyCr Thank you for reaching out! The database uses Snowflake, and the profiler was run manually. We have not specified any ingestion filters. Please kindly check the other details in our Slack conversation.

[TeddyCr]

Thanks @tanigawa-haruka by manually do you mean via the yaml config file? If so could you share it please (please maks sensitive info)

[tanigawa-haruka]

@TeddyCr I'm doing it manually from the OpenMetadata screen. I'm not using any particular file to run it. The Profiler Ingestion settings are as shown in the image.

[TeddyCr]

And you have the same error with the latest version right?

[tanigawa-haruka]

@TeddyCr Yes, it happens in version 1.5.6 as well.

[TeddyCr]

Have you redeployed the workflow after upgrading?

[tanigawa-haruka]

@TeddyCr Yes, the error still occurs when I redeploy. Just to be sure, I redeployed the Profiler Ingestion with the same settings earlier and ran it, but the error still occurred. I will also send you the error message from that time.

snowflake_profiler_f8U5bAP3-profiler.log

[TeddyCr]

@tanigawa-haruka can you try to create a new one and let me know if you it runs correctly?

[tanigawa-haruka]

@TeddyCr I got the same error when I created a new one. snowflake_profiler_uU6zs1Cp-profiler.log

[TeddyCr]

@tanigawa-haruka can you DM me on the OpenMetadata Slack channel so we can schedule a call to go over this?

[tanigawa-haruka]

@TeddyCr I’m very sorry. If possible, I would appreciate it if you could communicate with me via chat, as I do not speak English.

[TeddyCr]

@tanigawa-haruka can you try upgrading to 1.5.7. A user facing the same issue as you in lower version is not experiencing it in 1.5.7 anymore.

[tanigawa-haruka]

@TeddyCr Thank you for the advice!! I wasn't able to solve the problem by simply upgrading to version 1.5.7. However, I was able to resolve the issue by recreating the server and installing 1.5.7.

[TeddyCr]

Thank you for the information. Could you share more details regarding the steps you took to recreate the server?

[KazushiMatsuda]

@TeddyCr I will answer on behalf of Ms. Tanigawa. The server is created in AWS (os: aws linux2). The commands to be executed within linux are as follows

sudo amazon-linux-extras install -y docker

DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose

sudo usermod -aG docker $USER
exit

sudo systemctl start docker

mkdir openmetadata-docker && cd openmetadata-docker
curl -sL -o docker-compose.yml https://github.com/open-metadata/OpenMetadata/releases/download/1.5.7-release/dock

docker compose -f docker-compose.yml up --detach

[TeddyCr]

Thank you for the information. I'll close this ticket for now.