The metrics explorer doesn't seem to work

[ilisparrow]

🐛 Bug

Hello, The metric explorer always says no results when I am sure it should, and It doesn't work for any metric. It doesn when I look at metrics in a run.

To reproduce

Here are a few screenshots to show you what my problem is : When I go into a random run in the run explorer :

This is empty also, it used to be populated with the exact same code)

Expected behavior

Expected behavior is to get the plots displayed

Environment

• Aim Version 3.12.0
• Python 3.8.8
• pip 21.0.1
• Ubuntu 22.04

Additional context

the CLI prints ; No index was detected

Thank you very much in advance

[roubkar]

Hey @ilisparrow, thanks for reporting this, looking into it. Meanwhile could you please share example logs to help us reproduce this 🙏

[ilisparrow]

Thanks for the fast reply, Which logs do you me to share ? I track floats, integers and Text.

I sometimes get this error :

┌------------------------------------------------------------------------┐
                Aim UI collects anonymous usage analytics.
                        Read how to opt-out here:
    https://aimstack.readthedocs.io/en/latest/community/telemetry.html
└------------------------------------------------------------------------┘
Running Aim UI on repo <Repo#2677591850585676717 path=/home/main/.aim read_only=None>
Open http://127.0.0.1:43800
Press Ctrl+C to exit
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/main/anaconda3/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/middleware/cors.py", line 78, in __call__
    await self.app(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/routing.py", line 390, in handle
    await self.app(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/routing.py", line 55, in app
    await response(scope, receive, send)
  File "/home/main/anaconda3/lib/python3.8/site-packages/starlette/responses.py", line 286, in __call__
    raise RuntimeError(f"File at path {self.path} does not exist.")
RuntimeError: File at path /home/main/anaconda3/lib/python3.8/site-packages/aim_ui/build/4fca1a5251c0c653ac40.worker.js does not exist.

I use firefox if that's helpful

[roubkar]

Hey @ilisparrow, could you share a minimal code example to reproduce the issue related to not rendering plots in Metrics Explorer?

[roubkar]

Hey @ilisparrow, we've been able to reproduce the issue related to missing data in the Runs Page table and currently working on the fix.

As for not rendering the plots having a hard time reproducing it, could you try to rerun the aim up command and check if the issue is still there, if yes could you please share .aim folder (aim logs) so we can quickly debug and resolve the issue.

[ilisparrow]

While I was trying to make a simpler reproducable version, I woeked as it supposed to but then came back.

I need to force reinstall aim, and reinit the .aim rep

So I run this script as a test and it works when I run it the first time then it doesn't anymore.

from aim import Run


run = Run(
    experiment="ocr_orientation",
)

run["hparams"] = {
    "framework": "mmocr",
    "backbone_det": "param",
    "bbone_rec": "secondBB",
}


for i in range(10):
    run.track(
        i,
        name="well oriented from confidence",
    )
    run.track(
        1,
        name="AAAAA",
    )

[ilisparrow]

great ! thanks, for some reason, even when reinstall it and reset the .aim folder, It keeps in cache the old params, why is that ?

[roubkar]

How did you reset .aim folder, have you run aim init command?

[ilisparrow]

So : reinstall-> aim up : doesn't work reinstall-> aim init ->1 run -> aim up : doesn't work rm .aim-> aim init ->1 run -> aim up : doesn't work

So I can't figure out how does the bug happen. Here is the .aim folder : aim.zip

[ilisparrow]

How did you reset .aim folder, have you run aim init command?

I deleted it and just did aim init.

For some reason I feel like this bug is related to the fact that I still get the chips of the old params when I delete the folder and init it again

[ilisparrow]

Aha ! reinstall-> rm .aim ->aim init ->1 run -> aim up : doesn't work EDIT : Sorry it doesn't work, it was the parms explorer not the metrics one

[mihran113]

hey @ilisparrow. I was able to use metrics explorer successfully on the logs that you've provided. The fact that you're still getting the old params after resetting the .aim directory is pretty strange, and my guess here is that browser keeps some kind of cache actually. Could you try to clear the browser cache and see if that helps somehow?

[ilisparrow]

Thanks ! So it works on chrome/localhost and firefox/localhost and chrome/127.0.0.1 but not on firefox/127.0.0.1

You need to come back to the page for the new request to be generated.

I guess, that's already a win.

[mihran113]

@ilisparrow Pretty strange, I'm using firefox as well but on macOS, and it seems to be working fine in any case, will investigate further to see if can do anything to avoid it on ubuntu as well, meanwhile glad that there's some workaround. Please keep the issue open for the runs dashboard values rendering as that's a genuine issue on our side, and there's a work in progress already for the fix.

[ilisparrow]

No problem, I am glad I "contributed" a bit :). Thanks for being this quick !

[roubkar]

Hey @ilisparrow, we have just released v3.12.2 where the issue with Runs Page table values rendering is resolved. To upgrade please run pip install aim --upgrade

[ilisparrow]

Thanks, that was fast ! I did some more testing, and the problem seems to definitely be a firefox problem. I can't get the metrics explorer to display consistently in ff but in chrome I do.

Thanks again :)

[gorarakelyan]

closing this issue, as the fix was shipped with v3.12.2. @roubkar could you please submit a separate issue wrt firefox support to track the progress there.

[roubkar]

closing this issue, as the fix was shipped with v3.12.2. @roubkar could you please submit a separate issue wrt firefox support to track the progress there.

Sure!

[gloryVine]

I had the same problem and came here to ask for the issue to be reopened, since I am using Chromium and not Firefox. I ran a simple model with Python 3.9, aim 3.13.0 and got an empty Metrics Explorer. Ran the same code with Python 3.8, aim 3.10.3 and suddenly the metrics appeared.

Turns out that I ran aim up with the Python 3.8 environment in both cases. After switching to the Python 3.9 environment also for the aim up, I got my metrics also for the Python 3.9 model run. So if anybody else has this problem check this first.

[mihran113]

Hey @gloryVine! Thanks for the report. Can I ask you to provide a little bit more information about the case that is failing? (i.e. the exact python version, OS version, and if possible an example script to reproduce the issue.)