zenml icon indicating copy to clipboard operation
zenml copied to clipboard
verified publisher icon zenml-io

[BUG]: Error while fetching Device details. ERROR - Could not get IP location for W.X.Y.Z. (utils.py:395)

Open JustGitting opened this issue 7 months ago • 4 comments

Contact Details [Optional]

No response

System Information

ZenML Client info:

(.venv) PS C:\Users\user1\Documents\Projects\ZenML\zenml-io\examples\quickstart> zenml info -a -s

The current repo active project is no longer available.
Setting the repo active project to 'default'.
The current repo active stack is no longer available. Resetting the active stack to default.
ZENML_LOCAL_VERSION: 0.82.0
ZENML_SERVER_VERSION: 0.82.0
ZENML_SERVER_DATABASE: sqlite
ZENML_SERVER_DEPLOYMENT_TYPE: other
ZENML_CONFIG_DIR: C:\Users\user1\AppData\Roaming\zenml
ZENML_LOCAL_STORE_DIR: C:\Users\user1\AppData\Roaming\zenml\local_stores
ZENML_SERVER_URL: sqlite:///C:\Users\user1\AppData\Roaming\zenml\local_stores\default_zen_store/zenml.db
ZENML_ACTIVE_REPOSITORY_ROOT: C:\Users\user1\Documents\Projects\ZenML\zenml-io\examples\quickstart
PYTHON_VERSION: 3.11.2
ENVIRONMENT: native
SYSTEM_INFO: {'os': 'windows', 'windows_version_release': '10', 'windows_version': '10.0.19045', 'windows_version_service_pack': 'SP0', 'windows_version_os_type': 'Multiprocessor Free'}
ACTIVE_PROJECT: default
ACTIVE_STACK: default
ACTIVE_USER: default
TELEMETRY_STATUS: disabled
ANALYTICS_CLIENT_ID: CLIENT-9897d4e66d09
ANALYTICS_USER_ID: USER-a68951561ff9
ANALYTICS_SERVER_ID: CLIENT-9897d4e66d09
INTEGRATIONS: ['airflow', 'bitbucket', 'kaniko', 'mlflow', 'numpy', 'pandas', 'pigeon', 'pillow', 's3', 'scipy', 'sklearn']
PACKAGES: {'certifi': '2025.4.26', 'jsonschema-specifications': '2025.4.1', 'fsspec': '2025.3.2', 's3fs': '2025.3.2', 'pytz': '2025.2', 'tzdata': '2025.2', 'pywin32': '310', 'setuptools': '65.5.0', 
'cryptography': '44.0.3', 'pyzmq': '26.4.0', 'attrs': '25.3.0', 'pip': '25.1.1', 'webcolors': '24.11.1', 'packaging': '24.2', 'argon2-cffi': '23.1.0', 'argon2-cffi-bindings': '21.2.0', 'isoduration':   
'20.11.0', 'pyarrow': '19.0.1', 'websockets': '15.0.1', 'rich': '14.0.0', 'pillow': '11.2.1', 'ipython': '9.2.0', 'jupyter-client': '8.6.3', 'importlib-metadata': '8.6.1', 'click': '8.1.7', 
'ipywidgets': '8.1.7', 'nbconvert': '7.16.6', 'overrides': '7.7.0', 'notebook': '7.4.2', 'configparser': '7.2.0', 'docker': '7.1.0', 'psutil': '7.0.0', 'protobuf': '6.30.2', 'ipykernel': '6.29.5',      
'multidict': '6.4.3', 'tornado': '6.4.2', 'bleach': '6.2.0', 'pyyaml': '6.0.2', 'traitlets': '5.14.3', 'nbformat': '5.10.4', 'jupyter-core': '5.7.2', 'cachetools': '5.5.2', 'decorator': '5.2.1',        
'tldextract': '5.1.3', 'ipinfo': '5.1.1', 'smmap': '5.0.2', 'fonttools': '4.57.0', 'jsonschema': '4.23.0', 'beautifulsoup4': '4.13.4', 'typing-extensions': '4.13.2', 'rsa': '4.9.1', 'anyio': '4.9.0',   
'jupyterlab': '4.4.2', 'platformdirs': '4.3.8', 'widgetsnbextension': '4.0.14', 'gitdb': '4.0.12', 'bcrypt': '4.0.1', 'zipp': '3.21.0', 'filelock': '3.18.0', 'aiohttp': '3.11.18', 'orjson': '3.10.18',  
'matplotlib': '3.10.1', 'idna': '3.10', 'markdown': '3.8', 'threadpoolctl': '3.6.0', 'graphene': '3.4.3', 'charset-normalizer': '3.4.2', 'python-json-logger': '3.3.0', 'graphql-core': '3.2.6', 
'pyparsing': '3.2.3', 'greenlet': '3.2.1', 'graphql-relay': '3.2.0', 'gitpython': '3.1.44', 'jinja2': '3.1.6', 'mistune': '3.1.3', 'werkzeug': '3.1.3', 'flask': '3.1.0', 'prompt-toolkit': '3.0.51',     
'jupyterlab-widgets': '3.0.15', 'markupsafe': '3.0.2', 'waitress': '3.0.2', 'asttokens': '3.0.0', 'jsonpointer': '3.0.0', 'markdown-it-py': '3.0.0', 'google-auth': '2.40.0', 'pydantic-core': '2.33.0',  
'requests': '2.32.3', 'jupyterlab-server': '2.27.3', 'aiobotocore': '2.22.0', 'mlflow': '2.22.0', 'mlflow-skinny': '2.22.0', 'pycparser': '2.22', 'fastjsonschema': '2.21.1', 'pygments': '2.19.1',       
'babel': '2.17.0', 'jupyter-server': '2.15.0', 'pydantic': '2.11.1', 'pydantic-settings': '2.9.1', 'types-python-dateutil': '2.9.0.20241206', 'python-dateutil': '2.9.0.post0', 'pyjwt': '2.7.0',         
'soupsieve': '2.7', 'aiohappyeyeballs': '2.6.1', 'urllib3': '2.4.0', 'jupyter-lsp': '2.2.5', 'numpy': '2.2.5', 'pandas': '2.2.3', 'cloudpickle': '2.2.1', 'executing': '2.2.0', 'itsdangerous': '2.2.0',  
'requests-file': '2.1.0', 'sqlalchemy': '2.0.40', 'pywinpty': '2.0.15', 'async-lru': '2.0.5', 'boto3': '1.37.3', 'botocore': '1.37.3', 'opentelemetry-api': '1.32.1', 'opentelemetry-sdk': '1.32.1',      
'yarl': '1.20.0', 'wrapt': '1.17.2', 'cffi': '1.17.1', 'six': '1.17.0', 'scipy': '1.15.2', 'python-rapidjson': '1.14', 'blinker': '1.9.0', 'distro': '1.9.0', 'debugpy': '1.8.14', 'send2trash': '1.8.3', 
'alembic': '1.8.1', 'websocket-client': '1.8.0', 'passlib': '1.7.4', 'scikit-learn': '1.6.1', 'frozenlist': '1.6.0', 'nest-asyncio': '1.6.0', 'fqdn': '1.5.1', 'pandocfilters': '1.5.1', 'joblib':        
'1.5.0', 'kiwisolver': '1.4.8', 'argparse': '1.4.0', 'tinycss2': '1.4.0', 'mako': '1.3.10', 'aiosignal': '1.3.2', 'contourpy': '1.3.2', 'sniffio': '1.3.1', 'arrow': '1.3.0', 'uri-template': '1.3.0',    
'deprecated': '1.2.18', 'ipython-pygments-lexers': '1.1.1', 'pymysql': '1.1.1', 'python-dotenv': '1.1.0', 'httpcore': '1.0.9', 'watchfiles': '1.0.5', 'jmespath': '1.0.1', 'fastapi': '0.115.8', 'zenml': 
'0.82.0', 'opentelemetry-semantic-conventions': '0.53b1', 'databricks-sdk': '0.52.0', 'starlette': '0.45.3', 'sqlalchemy-utils': '0.41.2', 'referencing': '0.36.2', 'uvicorn': '0.34.2', 'httpx':         
'0.28.1', 'rpds-py': '0.24.0', 'prometheus-client': '0.21.1', 'jedi': '0.19.2', 'terminado': '0.18.1', 'h11': '0.16.0', 'cycler': '0.12.1', 'aioitertools': '0.12.0', 'json5': '0.12.0', 'jupyter-events':'0.12.0', 's3transfer': '0.11.3', 'nbclient': '0.10.2', 'parso': '0.8.4', 'aws-profile-manager': '0.7.3', 'defusedxml': '0.7.1', 'annotated-types': '0.7.0', 'httptools': '0.6.4', 'stack-data': '0.6.3', 
'pyasn1': '0.6.1', 'jupyter-server-terminals': '0.5.3', 'sqlparse': '0.5.3', 'webencodings': '0.5.1', 'colorama': '0.4.6', 'pyasn1-modules': '0.4.2', 'typing-inspection': '0.4.0', 'propcache': '0.3.1', 
'jupyterlab-pygments': '0.3.0', 'secure': '0.3.0', 'wcwidth': '0.2.13', 'notebook-shim': '0.2.4', 'pure-eval': '0.2.3', 'comm': '0.2.2', 'matplotlib-inline': '0.1.7', 'rfc3339-validator': '0.1.4',      
'mdurl': '0.1.2', 'rfc3986-validator': '0.1.1', 'python-multipart': '0.0.20', 'sqlmodel': '0.0.18'}

CURRENT STACK

Name: default
ID: X-e531c5de8cf7

ORCHESTRATOR: default

Name: default
ID: Y-ff76e274cf5f
Type: orchestrator
Flavor: local
Configuration: {}

ARTIFACT_STORE: default

Name: default
ID: Z-4dade805952d
Type: artifact_store
Flavor: local
Configuration: {}

What happened?

The ZenML client can log into the self-hosted remote ZenML server using the default "admin" user. Where I've successfully run the example pipeline from zenml.io examples.

However, I get "Error while fetching Device details." in the browser window when authorizing the same computer with the "user1" account (after logging out of the admin account) "user1" also has admin rights.

Reproduction steps

ZenML server: 192.168.100.165 GNU/Linux Devuan running docker compose zenmldocker/zenml-server

Client: 192.168.1.200 Windows 10 Enterprise VSCodium, python virtual environment, zenml 0.82.0

On the client machine (Windows 10 Enterprise), I connect to the remote ZenML server as below.

(.venv) PS C:\Users\user1\Documents\Projects\ZenML\zenml-io\examples\quickstart> zenml login 'http://192.168.100.165:8080' --refresh
Authenticating to ZenML server 'http://192.168.100.165:8080' using the web login...
If your browser did not open automatically, please open the following URL into your browser to proceed with the authentication:

http://192.168.100.165:8080/devices/verify?device_id=DEVICE-5f5505b0fb56&user_code=USER-246d47d

The browser window appears and I login as "user1", but I get the error:

Error while fetching Device details

Checking the ZenML server logs I found the following errors (cruft cut out for easier reading).

zenml server logs:

$ sudo docker compose -p zenml logs zenml

INFO:     127.0.0.1:38830 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55084 - "GET / HTTP/1.1" 200 OK
...
2025-05-08 03:09:29,168 - zenml.zen_server.utils - ERROR - Could not get IP location for 192.168.1.200. (utils.py:395)
Traceback (most recent call last):
  File "/opt/venv/lib/python3.11/site-packages/zenml/zen_server/utils.py", line 390, in get_ip_location
    details.city,
    ^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/ipinfo/details.py", line 16, in __getattr__
    raise AttributeError(f"{attr} is not a valid attribute of Details")
AttributeError: city is not a valid attribute of Details

INFO:     192.168.1.200:61035 - "POST /api/v1/device_authorization HTTP/1.1" 200 OK
...
2025-05-08 03:09:30,268 - zenml.zen_server.auth - ERROR - Authentication error: device with client ID CLIENT-9897d4e66d09 is pending. (auth.py:563)
...
2025-05-08 03:09:30,832 - asyncio - DEBUG - Returning static file: favicon.ico (zen_server_api.py:568)
...
2025-05-08 03:09:35,295 - zenml.zen_server.auth - ERROR - Authentication error: device with client ID CLIENT-9897d4e66d09 is pending. (auth.py:563)
INFO:     192.168.1.200:61051 - "POST /api/v1/login HTTP/1.1" 400 Bad Request
INFO:     127.0.0.1:47262 - "GET / HTTP/1.1" 200 OK
...
2025-05-08 03:09:40,321 - zenml.zen_server.auth - ERROR - Authentication error: device with client ID CLIENT-9897d4e66d09 is pending. (auth.py:563)

INFO:     192.168.1.200:61059 - "POST /api/v1/login HTTP/1.1" 400 Bad Request
INFO:     192.168.1.200:61064 - "POST /api/v1/login HTTP/1.1" 200 OK
INFO:     192.168.1.200:61064 - "GET /assets/page-DHnkTzqs.js HTTP/1.1" 200 OK
INFO:     192.168.1.200:61064 - "GET /assets/EmptyState-M1jafpg6.js HTTP/1.1" 200 OK
INFO:     192.168.1.200:61065 - "GET /assets/Error-vBjUYjb-.js HTTP/1.1" 200 OK
INFO:     192.168.1.200:61066 - "GET /assets/check-circle-mvyzYvIW.js HTTP/1.1" 200 OK

2025-05-08 03:09:44,934 - zenml.zen_server.auth - INFO - Got token from cookie (auth.py:1061)

INFO:     192.168.1.200:61066 - "GET /api/v1/info HTTP/1.1" 200 OK

2025-05-08 03:09:44,959 - zenml.zen_server.utils - ERROR - API error (utils.py:311)
Traceback (most recent call last):
  File "/opt/venv/lib/python3.11/site-packages/zenml/zen_server/utils.py", line 301, in decorated
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/zenml/zen_server/routers/devices_endpoints.py", line 129, in get_authorization_device
    raise KeyError(
KeyError: 'Unable to get device with ID DEVICE-5f5505b0fb56: No device with this ID found.'

If I log "user1" out and try to re-authorise the client using the "admin" user, then it is successful.

Is there anything I need to set for a user account to authenticate a zenml client?

Why is it trying to get the IP "location"?

Relevant log output

See above.

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

JustGitting avatar May 08 '25 04:05 JustGitting

@JustGitting Fetching the IP location is displayed in the UI as a safety measure, so you don't accidentally log in a client machine that isn't actually you. When it fails, the exception is logged but shouldn't actually interfere with the login process.

Just to confirm, zenml login ... works for you if you log in with the admin user in the browser tab that gets opened, correct?

schustmi avatar May 08 '25 07:05 schustmi

Just to confirm, zenml login ... works for you if you log in with the admin user in the browser tab that gets opened, correct?

Yes, I can authorize the client with the admin account via the browser.

JustGitting avatar May 08 '25 08:05 JustGitting

Yep found the issue, I have a quite involved workaround for you in the meantime, in case this is urgent: When trying to login to a server with multiple accounts from the same client machine, you'll need to delete the currently authorized device. Concretely:

  • Login to your admin account.
  • Run zenml authorized-device list. There should be a single device in there.
  • Run zenml authorized-device delete <ID>.
  • Run zenml logout --clear
  • run zenml login <URL> and use the other account to login

schustmi avatar May 08 '25 08:05 schustmi

@schustmi The workaround does the trick.

JustGitting avatar May 09 '25 05:05 JustGitting

@JustGitting thank your for reporting this. We will handle this issue over here. So, I am closing this one. Feel free to reach out again if you encounter any other problems.

bcdurak avatar Sep 23 '25 15:09 bcdurak