index page does not load

[ariG23498]

Description

wandb local creates the container but the index page localhost:8080 does not load. I have tried wandb local --upgrade and wandb local -e LOCAL_RESTORE=true, but both the commands does not help.

Follow

• wandb local
• docker logs -f wandb-local

*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/01_enable-services.sh...
*** Copying services to runit
*** Copying jobber template
*** Enabling production mode
*** Running /etc/my_init.d/02_load-settings.sh...
*** Loading settings...
*** Booting runit daemon...
*** Runit started as PID 36
*** Starting wandb servers...
*** Migrating database...
panic: Dirty database version 1. Fix and force version.

goroutine 1 [running]:
main.main()
	/mnt/ramdisk/core/services/gorilla/cmd/migrate/main.go:115 +0xb05
*** All services started
*** Access W&B at http://localhost:8080
!!! Unable to start backend, see logs at /var/logs.  Trying again in 1 minute
!!! Set the environment variable LOCAL_RESTORE=true if you're unable to login to the admin console
*** Starting wandb servers...
*** All services started
*** Access W&B at http://localhost:8080
*** Starting wandb servers...
*** All services started
*** Access W&B at http://localhost:8080
!!! Unable to start backend, see logs at /var/logs.  Trying again in 1 minute
!!! Set the environment variable LOCAL_RESTORE=true if you're unable to login to the admin console
^C

Versions

OS: Ubuntu 20.04.2 LTS wandb: wandb, version 0.10.30 docker: Docker version 20.10.6, build 370c289

Is this a problem with the panic: Dirty database version 1. Fix and force version.?

[TUMAAA]

I wonder why this question is not answered :(

[vanpelt]

Apologies for not answering this sooner. The error indicates we were unable to migrate the database. If you're working with a fresh wandb/local install without any data in it, you can remove the persistent volume and start it again (warning this will remove all data):

docker stop wandb-local
docker volume rm wandb
wandb local

If you get this error with a database that already has data in it that you want to keep, the process to fix the instance is more involved and would require a screenshare / direct support from our team.

[carloalbertobarbano]

@vanpelt same happens to me. same error, removing volume is not doing anything. wandb-local has become unusable

[vanpelt]

@carloalbertobarbano if you're getting panic: Dirty database version error or the entity already exists error? I'll respond to the other ticket, but if you can share the exact error you're showing we can get you back up and running.

[carloalbertobarbano]

Sure, maybe let's keep the discussion in #39 so it is easier to follow?

[YarrDOpanas]

Hello everyone! The same error exists on my side

docker stop wandb-local
docker volume rm wandb
wandb local

didn't help. After executing this command docker logs -f wandb-local gives me

*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/01_enable-services.sh...
*** Copying services to runit
*** Copying jobber template
*** Enabling production mode
*** Running /etc/my_init.d/02_load-settings.sh...
*** Loading settings...
2022/04/13 11:57:17 Created default user
2022/04/13 11:57:17 Generating new session key for auth...
2022/04/13 11:57:17 Generating new certificate and key for auth...
*** Booting runit daemon...
*** Runit started as PID 57
*** Setting up mysql database...
*** Starting wandb servers...
*** Configuring minio...
Bucket created successfully `local/local-files`.
Successfully added arn:minio:sqs:wandb-local:_:redis
*** Migrating database...

and on authorization page I can see

Loading is taking longer than expected. Check stdout, or the system logs at /var/log for error messages. You can restart your server with the environment variable LOCAL_RESTORE=true to regain access if you're unable to login.

Versions

OS: Ubuntu 20.04.3 LTS wandb: wandb, version 0.12.14 docker:Docker version 20.10.14, build a224086

[vanpelt]

Hey @YarrDOpanas that message indicates the migrations are have not yet completed. This generally means you have a very poor performing file system. This can be for a number of reasons. Can you share the details about your OS and architecture you're running this from? I.E linux x86 or Mac arm64 etc

[YarrDOpanas]

Hey, @vanpelt , thank you for a such quick reply! I did wandb local --upgrade . Not sure, if it was the reason, but after that it started working) BTW, I also had to visit http://localhost:8080/logout after removing wandb-local docker volume .

Architecture on the server where wandb-local is running is Linux SilverBulletGeneral 5.11.0-1028-azure #31~20.04.2-Ubuntu SMP Tue Jan 18 08:46:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux . And port forwarding is enabled to my browser.

[djaym7]

Hey, @vanpelt , thank you for a such quick reply! I did wandb local --upgrade . Not sure, if it was the reason, but after that it started working) BTW, I also had to visit http://localhost:8080/logout after removing wandb-local docker volume .

Architecture on the server where wandb-local is running is Linux SilverBulletGeneral 5.11.0-1028-azure #31~20.04.2-Ubuntu SMP Tue Jan 18 08:46:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux . And port forwarding is enabled to my browser.

This worked for me too, now after running below three commands, l can change port and it works docker stop wandb-local docker volume rm wandb wandb local --upgrade

[sydholl]

WandB Internal User commented: carloalbertobarbano commented: @vanpelt same happens to me. same error, removing volume is not doing anything. wandb-local has become unusable

[sydholl]

WandB Internal User commented: carloalbertobarbano commented: Sure, maybe let's keep the discussion in #39 so it is easier to follow?

[sydholl]

WandB Internal User commented: YarrDOpanas commented: Hello everyone! The same error exists on my side

docker stop wandb-local
docker volume rm wandb
wandb local

didn't help. After executing this command docker logs -f wandb-local gives me

*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/01_enable-services.sh...
*** Copying services to runit
*** Copying jobber template
*** Enabling production mode
*** Running /etc/my_init.d/02_load-settings.sh...
*** Loading settings...
2022/04/13 11:57:17 Created default user
2022/04/13 11:57:17 Generating new session key for auth...
2022/04/13 11:57:17 Generating new certificate and key for auth...
*** Booting runit daemon...
*** Runit started as PID 57
*** Setting up mysql database...
*** Starting wandb servers...
*** Configuring minio...
Bucket created successfully `local/local-files`.
Successfully added arn:minio:sqs:wandb-local:_:redis
*** Migrating database...

and on authorization page I can see

Loading is taking longer than expected. Check stdout, or the system logs at /var/log for error messages. You can restart your server with the environment variable LOCAL_RESTORE=true to regain access if you're unable to login.

Versions

OS: Ubuntu 20.04.3 LTS wandb: wandb, version 0.12.14 docker:Docker version 20.10.14, build a224086

[sydholl]

WandB Internal User commented: vanpelt commented: Hey @YarrDOpanas that message indicates the migrations are have not yet completed. This generally means you have a very poor performing file system. This can be for a number of reasons. Can you share the details about your OS and architecture you're running this from? I.E linux x86 or Mac arm64 etc

[sydholl]

WandB Internal User commented: YarrDOpanas commented: Hey, @vanpelt , thank you for a such quick reply! I did wandb local --upgrade . Not sure, if it was the reason, but after that it started working) BTW, I also had to visit http://localhost:8080/logout after removing wandb-local docker volume .

Architecture on the server where wandb-local is running is Linux SilverBulletGeneral 5.11.0-1028-azure #31~20.04.2-Ubuntu SMP Tue Jan 18 08:46:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux . And port forwarding is enabled to my browser.

[sydholl]

WandB Internal User commented: djaym7 commented:

Hey, @vanpelt , thank you for a such quick reply! I did wandb local --upgrade . Not sure, if it was the reason, but after that it started working) BTW, I also had to visit http://localhost:8080/logout after removing wandb-local docker volume .

Architecture on the server where wandb-local is running is Linux SilverBulletGeneral 5.11.0-1028-azure #31~20.04.2-Ubuntu SMP Tue Jan 18 08:46:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux . And port forwarding is enabled to my browser.

This worked for me too, now after running below three commands, l can change port and it works docker stop wandb-local docker volume rm wandb wandb local --upgrade

[pamdla]

# wandb local --upgrade
wandb: WARNING `wandb local` has been replaced with `wandb server start`.

my os: MBP MacOS 12.6.3 wandb: 0.17.1

not working here ...