Metaflow-UI won't connect

[AbdAlRahman-Odeh-99]

Description

When I run the docker image of the UI and redirect it to port 3000, the terminal shows that the processes are being started but the UI shows waiting for connection. Expected behavior:

• The UI would start-up and I would be able to submit DAGs. Actual behavior:
• The UI keeps waiting for connection and the submitted tags shows [Error111] Cnnection refused Reproduces how often:
• Everytime

Versions

• Metaflow 2.5 & 2.7
• Python 3.8
• Ubuntu 20.04

Additional Information

• The DAG works fine when submitted without the UI parameters

[obgibson]

Can you open the browser developer tools and take a look at what errors you get from the requests from the UI to the server?

On Aug 7, 2022, at 6:49 PM, AbdAlRahman-Odeh-99 @.***> wrote:

 Description

When I run the docker image of the UI and redirect it to port 3000, the terminal shows that the processes are being started but the UI shows waiting for connection. Expected behavior:

The UI would start-up and I would be able to submit DAGs. Actual behavior: The UI keeps waiting for connection and the submitted tags shows [Error111] Cnnection refused Reproduces how often: Everytime Versions

Metaflow 2.5 & 2.7 Python 3.8 Ubuntu 20.04 Additional Information

The DAG works fine when submitted without the UI parameters

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[AbdAlRahman-Odeh-99]

Here is what the console shows.

[obgibson]

How are you running the metaflow-service? Are you following the instructions here https://github.com/Netflix/metaflow-service ? The metaflow-service needs to be running for the metaflow-ui to talk to.

[AbdAlRahman-Odeh-99]

Come to think of it, the metaflow-service always kept giving me permission denied error. My installation was through: git clone https://github.com/Netflix/metaflow-service.git && cd metaflow-service docker-compose -f docker-compose.development.yml up

[obgibson]

What I can see here is an issue with docker. I looked around the internet and it seems like it is probably an issue with the permissions of your user - https://stackoverflow.com/questions/48957195/how-to-fix-docker-got-permission-denied-issue/48957722#48957722 might be a good way to solve the problem.

[AbdAlRahman-Odeh-99]

Thanks @obgibson, this fixed the issue of permission denied, but now there is anew issue with the metaflow backend server. I have removed and r-installed the metaflow-service folder but still got the same issue.

[AbdAlRahman-Odeh-99]

@obgibson Any update regarding this issue ?

[obgibson]

Take a look at https://github.com/netflix/metaflow-service#migration-service - there are instructions to first check what version of the schema you have, and to update it if yours is out of date.

[AbdAlRahman-Odeh-99]

Hello @obgibson, I have restarted the installation process, here are the steps that I have done: 1- git clone https://github.com/Netflix/metaflow-service.git 2- cd metaflow-service 3- pip3 install ./ 4- python3 -m services.metadata_service.server At step 4, the following error occurs:

/home/ceip2/metaflow-service/services/metadata_service/server.py:22: DeprecationWarning: loop argument is deprecated
  app = web.Application(loop=loop, middlewares=middlewares)
ERROR:AsyncPostgresDB:global:Exception occurred
Traceback (most recent call last):
  File "/home/ceip2/metaflow-service/services/data/postgres_async_db.py", line 78, in _init
    self.pool = await aiopg.create_pool(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 300, in from_pool_fill
    await self._fill_free_pool(False)
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 336, in _fill_free_pool
    conn = await connect(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 1225, in _connect
    await self._poll(self._waiter, self._timeout)  # type: ignore
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 881, in _poll
    await asyncio.wait_for(self._waiter, timeout)
  File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 788, in _ready
    state = self._conn.poll()
psycopg2.OperationalError: Connection refused
	Is the server running on that host and accepting TCP/IP connections?

ERROR:AsyncPostgresDB:global:Exception occurred
Traceback (most recent call last):
  File "/home/ceip2/metaflow-service/services/data/postgres_async_db.py", line 78, in _init
    self.pool = await aiopg.create_pool(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 300, in from_pool_fill
    await self._fill_free_pool(False)
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 336, in _fill_free_pool
    conn = await connect(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 1225, in _connect
    await self._poll(self._waiter, self._timeout)  # type: ignore
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 881, in _poll
    await asyncio.wait_for(self._waiter, timeout)
  File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 788, in _ready
    state = self._conn.poll()
psycopg2.OperationalError: Connection refused
	Is the server running on that host and accepting TCP/IP connections?

ERROR:AsyncPostgresDB:global:Exception occurred
Traceback (most recent call last):
  File "/home/ceip2/metaflow-service/services/data/postgres_async_db.py", line 78, in _init
    self.pool = await aiopg.create_pool(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 300, in from_pool_fill
    await self._fill_free_pool(False)
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 336, in _fill_free_pool
    conn = await connect(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 1225, in _connect
    await self._poll(self._waiter, self._timeout)  # type: ignore
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 881, in _poll
    await asyncio.wait_for(self._waiter, timeout)
  File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 788, in _ready
    state = self._conn.poll()
psycopg2.OperationalError: Connection refused
	Is the server running on that host and accepting TCP/IP connections?

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/ceip2/metaflow-service/services/metadata_service/server.py", line 52, in <module>
    main()
  File "/home/ceip2/metaflow-service/services/metadata_service/server.py", line 37, in main
    the_app = app(loop, DBConfiguration())
  File "/home/ceip2/metaflow-service/services/metadata_service/server.py", line 24, in app
    loop.run_until_complete(async_db._init(db_conf))
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/ceip2/metaflow-service/services/data/postgres_async_db.py", line 98, in _init
    raise e
  File "/home/ceip2/metaflow-service/services/data/postgres_async_db.py", line 78, in _init
    self.pool = await aiopg.create_pool(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 300, in from_pool_fill
    await self._fill_free_pool(False)
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/pool.py", line 336, in _fill_free_pool
    conn = await connect(
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 1225, in _connect
    await self._poll(self._waiter, self._timeout)  # type: ignore
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 881, in _poll
    await asyncio.wait_for(self._waiter, timeout)
  File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/ceip2/.local/lib/python3.8/site-packages/aiopg/connection.py", line 788, in _ready
    state = self._conn.poll()
psycopg2.OperationalError: Connection refused
	Is the server running on that host and accepting TCP/IP connections?

[obgibson]

Any luck with working this out? It looks to me like postgres is not running? If not, jump onto the slack channel where you can get help a bit more synchronously.

[AbdAlRahman-Odeh-99]

@obgibson Unfortunately, I had no luck solving this yet, and yes I think it is a problem with Postgres. Is there a way to run it using an alternative Database such as MySQL? Please provide me with the link for the slack channel.

[obgibson]

http://chat.metaflow.org/ - put your question in the #ask-metaflow channel.

I don't think that MySQL could be swapped in easily. Let's see if we can get postgres working for you.

[jaeyow]

Recently fixed in metaflow-service. https://github.com/Netflix/metaflow-service/pull/309

[AbdAlRahman-Odeh-99]

@obgibson @jaeyow Thank you for your support. Running the following steps solved it for me:

git clone https://github.com/Netflix/metaflow-service.git && cd metaflow-service
docker-compose -f docker-compose.development.yml build
docker-compose -f docker-compose.development.yml up
docker run -p 3000:3000 -e METAFLOW_SERVICE=http://localhost:8083/ asd8511tw/metaflow-ui