server icon indicating copy to clipboard operation
server copied to clipboard

Local Quick Start Guide doesn't work

Open Inquisitive-ME opened this issue 2 years ago • 14 comments
trafficstars

I am running wandb server on 1 computer and trying the quickstart example code on another computer all on my local network.

On the computer I am trying to run the quickstart guide on I cannot get wandb.init() to work in Python. The output I get is:

wandb: W&B API key is configured. Use `wandb login --relogin` to force relogin
wandb: Network error (ConnectionError), entering retry loop.
wandb: Network error (ConnectionError), entering retry loop.

I am running wandb login on the command line before running the Python scrip that calls wandb.init()

I am starting wandb server with wandb server start -e HOST=http://192.168.99.156:8080

and have tried the login with both wandb login --host http://192.168.99.165/ wandb login --host http://192.168.99.165:8080

and can access the webpage from the computer I am trying to run wandb.init() in Python on.

Inquisitive-ME avatar Mar 08 '23 16:03 Inquisitive-ME

Hi @Inquisitive-ME, still looking into this on our side. Can you try running wandb verify on the machine that is not running the Docker container?

Thank you, Nate

nate-wandb avatar Mar 09 '23 19:03 nate-wandb

It just appears to hang and I don't see any logs:

Default host selected: http://192.168.99.165
Find detailed logs for this test at: /tmp/tmphra6d1qd/wandb

Wandb verify works on the host machine:

Default host selected: http://localhost:8080
Find detailed logs for this test at: /tmp/tmp5pihk34z/wandb
Checking if logged in...................................................✅
Checking signed URL upload..............................................✅
Checking ability to send large payloads through proxy...................✅
Checking requests to base url...........................................❌
Connections are not made over https. SSL required for secure communications.
Checking requests made over signed URLs.................................❌
Signed URL requests not made over https. SSL is required for secure communications.
Checking CORs configuration of the bucket...............................❌
Your object store does not have a valid CORs configuration, you must allow GET and PUT to Origin: http://localhost:8080
Checking wandb package version is up to date............................❌
wandb version is not supported by your local installation. This could cause some issues. If you're having problems try: please run `pip install --upgrade wandb==0.13.9`
Checking logged metrics, saving and downloading a file..................✅
Checking artifact save and download workflows...........................✅

Inquisitive-ME avatar Mar 11 '23 00:03 Inquisitive-ME

@Inquisitive-ME I notice that on the non-host machine, it is trying to access http://192.168.99.165 but the port is not specified.

I believe you tried logging in with wandb login --host=http://192.168.99.165:8080 --relogin correct? If so, can you delete ~/.netrc and `~/.config/wandb/settings and then log in again?

Then when you run wandb login --host=http://192.168.99.165:8080 --relogin it should point to correct port.

Running wandb verify should print Default host selected: http://192.168.99.165:8080 at that point and I believe this will resolve the issues with the API not connecting as well

Thank you, Nate

nate-wandb avatar Mar 13 '23 15:03 nate-wandb

Hi @Inquisitive-ME, I just wanted to follow up and see if that helped or if the API is still not pointing to the correct port?

Thank you, Nate

nate-wandb avatar Mar 15 '23 14:03 nate-wandb

Hi I tried the above steps and get

Default host selected: http://192.168.99.165:8080

In the wandb verify output but it is still hung. When I cancel wandb verify it outputs

Aborted!
Checking if logged in...................................................

Inquisitive-ME avatar Mar 23 '23 00:03 Inquisitive-ME

I found that even if I login on the host machine with the host set to 192.168.99.165:8080 it does not work. wandb verify also hangs. The host has to be set to http://localhost:8080 for it to work

Inquisitive-ME avatar Mar 23 '23 01:03 Inquisitive-ME

@Inquisitive-ME on the non-host machine you are able to access the UI at 192.168.99.165:8080 in the browser, correct?

nate-wandb avatar Mar 24 '23 14:03 nate-wandb

@nate-wandb that is correct. I am able to access the UI at 192.168.99.165:8080 in the browser. But wandb verify is hanging and any attempt to connect with Python examples ends up in a retry loop

Inquisitive-ME avatar Mar 24 '23 15:03 Inquisitive-ME

@Inquisitive-ME that indicates the machine you're running wandb verify on can't connect to 192.168.99.165. This is likely due to firewall rules or other networking issues.

vanpelt avatar Mar 24 '23 22:03 vanpelt

Okay, it seems like it's an issue with the computer I was trying to run as host. Are there specific ports that wandb uses. I would guess that a port that wandb uses is in use already?

Inquisitive-ME avatar Mar 24 '23 22:03 Inquisitive-ME

WandB Internal User commented: Inquisitive-ME commented: I found that even if I login on the host machine with the host set to 192.168.99.165:8080 it does not work. wandb verify also hangs. The host has to be set to http://localhost:8080 for it to work

exalate-issue-sync[bot] avatar Apr 05 '23 18:04 exalate-issue-sync[bot]

WandB Internal User commented: Inquisitive-ME commented: Okay, it seems like it's an issue with the computer I was trying to run as host. Are there specific ports that wandb uses. I would guess that a port that wandb uses is in use already?

exalate-issue-sync[bot] avatar Apr 05 '23 18:04 exalate-issue-sync[bot]

Hi @Inquisitive-ME, wandb server will use 8080 by default but you can specify which port you would like the server to use by starting the container with the following command:

docker run --rm -d -v wandb:/vol -p <available port you would like to access the app on>:8080 --name wandb-local wandb/local

nate-wandb avatar Apr 05 '23 19:04 nate-wandb

Hi @Inquisitive-ME, I wanted to follow up and see if you were able to make this work or if you had any other questions?

nate-wandb avatar Apr 12 '23 14:04 nate-wandb