datahub icon indicating copy to clipboard operation
datahub copied to clipboard

Error while retrieving '/api/user' info from metabase

Open mr-woot opened this issue 2 years ago • 3 comments

Describe the bug I have setup metabase with admin credentials. The following error is observed while retrieving '/api/user' info from metabase.

To Reproduce Steps to reproduce the behavior:

  1. Go to 'Ingestion'
  2. Click on 'Create new source'
  3. Scroll down to 'Custom'
  4. Added config for metabase as following
source:
    type: metabase
    config:
        connect_uri: "${METABASE_CONNECT_URI}"
        username: "${METABASE_USERNAME}"
        password: "${METABASE_PASSWORD}"
        default_schema: public
sink:
    type: datahub-rest
    config:
        server: "${DATAHUB_REST_URL}"

  1. See error

Expected behavior

~~~~ Execution Summary ~~~~

RUN_INGEST - {'errors': [],
 'exec_id': '40c8a14c-b677-4495-823f-101068da0814',
 'infos': ['2022-06-30 04:31:57.670401 [exec_id=40c8a14c-b677-4495-823f-101068da0814] INFO: Starting execution for task with name=RUN_INGEST',
**///// long list of charts and dashboard queries**
           "              'metabase-user-8': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                  'https://metabase.apps.janio.asia/api/user/8'],\n"
           "              'metabase-user-79': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/79'],\n"
           "              'metabase-table-card__489': ['Unable to retrieve source table. Reason: 404 Client Error: Not Found for url: '\n"
           "                                           'https://metabase.apps.janio.asia/api/table/card__489'],\n"
           "              'metabase-table-card__1083': ['Unable to retrieve source table. Reason: 404 Client Error: Not Found for url: '\n"
           "                                            'https://metabase.apps.janio.asia/api/table/card__1083'],\n"
           "              'metabase-user-162': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/162'],\n"
           "              'metabase-user-16': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/16'],\n"
           "              'metabase-user-239': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/239'],\n"
           "              'metabase-user-32': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/32'],\n"
           "              'metabase-user-354': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/354'],\n"
           "              'metabase-user-168': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/168'],\n"
           "              'metabase-user-40': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/40'],\n"
           "              'metabase-user-93': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/93'],\n"
           "              'metabase-user-252': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/252'],\n"
           "              'metabase-user-26': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/26'],\n"
           "              'metabase-user-39': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/39'],\n"
           "              'metabase-user-259': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/259'],\n"
           "              'metabase-user-173': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/173'],\n"
           "              'metabase-user-114': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/114'],\n"
           "              'metabase-user-268': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/268'],\n"
           "              'metabase-user-71': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/71'],\n"
           "              'metabase-user-147': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/147'],\n"
           "              'metabase-user-241': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/241'],\n"
           "              'metabase-user-98': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/98'],\n"
           "              'metabase-user-38': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/38'],\n"
           "              'metabase-user-41': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/41'],\n"
           "              'metabase-user-18': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/18'],\n"
           "              'metabase-user-17': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/17'],\n"
           "              'metabase-user-37': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/37'],\n"
           "              'metabase-user-36': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/36'],\n"
           "              'metabase-user-45': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/45'],\n"
           "              'metabase-user-11': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/11'],\n"
           "              'metabase-user-125': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/125'],\n"
           "              'metabase-user-131': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/131'],\n"
           "              'metabase-user-152': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/152'],\n"
           "              'metabase-user-29': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/29'],\n"
           "              'metabase-user-341': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/341'],\n"
           "              'metabase-user-19': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/19'],\n"
           "              'metabase-user-15': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/15'],\n"
           "              'metabase-user-99': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/99'],\n"
           "              'metabase-table-card__2516': ['Unable to retrieve source table. Reason: 404 Client Error: Not Found for url: '\n"
           "                                            'https://metabase.apps.janio.asia/api/table/card__2516'],\n"
           "              'metabase-user-65': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/65'],\n"
           "              'metabase-user-290': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/290'],\n"
           "              'metabase-user-277': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/277'],\n"
           "              'metabase-user-35': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/35'],\n"
           "              'metabase-user-57': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/57'],\n"
           "              'metabase-user-28': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                   'https://metabase.apps.janio.asia/api/user/28'],\n"
           "              'metabase-table-card__494': ['Unable to retrieve source table. Reason: 404 Client Error: Not Found for url: '\n"
           "                                           'https://metabase.apps.janio.asia/api/table/card__494'],\n"
           "              'metabase-user-139': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/139'],\n"
           "              'metabase-user-292': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/292'],\n"
           "              'metabase-user-287': ['Unable to retrieve User info. Reason: 404 Client Error: Not Found for url: '\n"
           "                                    'https://metabase.apps.janio.asia/api/user/287']},\n"
           " 'cli_version': '0.8.38.2',\n"
           " 'cli_entry_location': '/tmp/datahub/ingest/venv-40c8a14c-b677-4495-823f-101068da0814/lib/python3.9/site-packages/datahub/__init__.py',\n"
           " 'py_version': '3.9.9 (main, Dec 21 2021, 10:03:34) \\n[GCC 10.2.1 20210110]',\n"
           " 'py_exec_path': '/tmp/datahub/ingest/venv-40c8a14c-b677-4495-823f-101068da0814/bin/python3',\n"
           " 'os_details': 'Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.31'}\n"
           'Sink (datahub-rest) report:\n'
           "{'records_written': 5851,\n"
           " 'warnings': [],\n"
           " 'failures': [],\n"
           " 'downstream_start_time': datetime.datetime(2022, 6, 30, 4, 32, 48, 671457),\n"
           " 'downstream_end_time': datetime.datetime(2022, 6, 30, 4, 59, 23, 887262),\n"
           " 'downstream_total_latency_in_seconds': 1595.215805,\n"
           " 'gms_version': 'v0.8.39'}\n"
           '\n'
           'Pipeline finished with 107 failures in source producing 5851 workunits\n'
           '\x1b[31m❗Client-Server Incompatible❗\x1b[0m \x1b[36mYour client version 0.8.38.2 is older than your server version 0.8.39. Upgrading the '
           'cli to 0.8.39 is recommended.\n'
           '\x1b[0m \x1b[36m➡️  Upgrade via "pip install \'acryl-datahub==0.8.39\'"\x1b[0m\n',
           "2022-06-30 04:59:26.159938 [exec_id=40c8a14c-b677-4495-823f-101068da0814] INFO: Failed to execute 'datahub ingest'",
           '2022-06-30 04:59:26.160258 [exec_id=40c8a14c-b677-4495-823f-101068da0814] INFO: Caught exception EXECUTING '
           'task_id=40c8a14c-b677-4495-823f-101068da0814, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 121, in execute_task\n'
           '    self.event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 89, in run_until_complete\n'
           '    return f.result()\n'
           '  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
           '    raise self._exception\n'
           '  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
           '    result = coro.send(None)\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 115, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}
Execution finished with errors.

Desktop (please complete the following information):

  • OS: CentOS 8

mr-woot avatar Jun 30 '22 05:06 mr-woot

This issue is stale because it has been open for 15 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

github-actions[bot] avatar Aug 02 '22 02:08 github-actions[bot]

Hello @mr-woot The described error also happens in my scenario. However, I found that the error generating user was disabled. So the solution would be to change /api/user/:id to /api/user/ with the status=all filter.

Here is the endpoint documentation: https://github.com/metabase/metabase/blob/master/docs/api/user.md#get-apiuser

joaoluizjoaquim avatar Aug 18 '22 17:08 joaoluizjoaquim

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

github-actions[bot] avatar Sep 18 '22 02:09 github-actions[bot]

This issue was closed because it has been inactive for 30 days since being marked as stale.

github-actions[bot] avatar Oct 19 '22 02:10 github-actions[bot]