clearml-agent init fails. Error: could not verify credentials:

[vineetsharma14]

Hello There,

I have hosted the ClearML Server on spare computer (Windows 10 - Intel i3 with 12 GBs of RAM, 8 GBs allocated to Docker Desktop). However, when I try to add a worker (Windows 11 Laptop with RTX 2060 GPU) using the "clearml-agent init" command; it fails with the message "Error: could not verify credentials:"

Below is the log generated on the console with the clearml-agent init command

CLEARML-AGENT setup process

Please create new clearml credentials through the settings page in your clearml-server web app, or create a free account at https://app.clear.ml/settings/webapp-configuration

In the settings > workspace page, press "Create new credentials", then press "Copy to clipboard".

Paste copied configuration here: api { web_server: http://192.168.0.244:8080 api_server: http://192.168.0.244:8008 files_server: http://192.168.0.244:8081 # Acer300 credentials { "access_key" = "GHZDS5FAO218MA3LQDNE" "secret_key" = "tJb354Q39sOX6J6HdI23qKV9kNQ6vvBYC9o6xEvSlsiFYk8j0K" } } Detected credentials key="GHZDS5FAO218MA3LQDNE" secret="tJb3***" WEB Host configured to: [http://192.168.0.244:8080] API Host configured to: [http://192.168.0.244:8008] File Store Host configured to: [http://192.168.0.244:8081]

ClearML Hosts configuration: Web App: http://192.168.0.244:8080 API: http://192.168.0.244:8008 File Store: http://192.168.0.244:8081

Verifying credentials ... Error: could not verify credentials: key=GHZDS5FAO218MA3LQDNE secret=tJb354Q39sOX6J6HdI23qKV9kNQ6vvBYC9o6xEvSlsiFYk8j0K Enter user access key: Enter user secret: Verifying credentials ... Using built-in ClearML default key/secret Error: could not verify credentials: key= secret= Exiting setup without creating configuration file

I have checked the issues on the ClearML Repo. ClearML Server and ClearML Agent repos and have tried the following things -

1. Enabled Web Authentication on the ClearML Server with a fixed username and password.

2. Tried to reach the ClearML Web Server with the curl command as below. On both the occasions - I was able to see the HTML returned on the terminal.

curl http://192.168.0.244:8080/auth.login

and

curl -u GHZDS5FAO218MA3LQDNE:tJb354Q39sOX6J6HdI23qKV9kNQ6vvBYC9o6xEvSlsiFYk8j0K http://192.168.0.244:8080/auth.login

3. Tried the CURL Command to the API Server with the Access and the Secret Key as below

curl -u GHZDS5FAO218MA3LQDNE:tJb354Q39sOX6J6HdI23qKV9kNQ6vvBYC9o6xEvSlsiFYk8j0K http://192.168.0.244:8008/auth.login

It returned the following log on the terminal

{"meta":{"id":"f9b8cb6453a44a0db700272832af5608","trx":"f9b8cb6453a44a0db700272832af5608","endpoint":{"name":"auth.login","requested_version":"2.21","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhdXRoX3R5cGUiOiJCZWFyZXIiLCJpZGVudGl0eSI6eyJ1c2VyIjoiNGVhNzdmOTVmNDYyZDFhZjBmMDEyNjRlOGNjZDg0MDUiLCJ1c2VyX25hbWUiOiJWaW5lZXQiLCJjb21wYW55X25hbWUiOiJjbGVhcm1sIiwiY29tcGFueSI6ImQxYmQ5MmEzYjAzOTQwMGNiYWZjNjBhN2E1YjFlNTJiIiwicm9sZSI6InVzZXIifSwiZW52IjoiPHVua25vd24-IiwiaWF0IjoxNjcwMDg3MjU4LCJleHAiOjE2NzI2NzkyNTgsImFwaV92ZXJzaW9uIjoiMi4yMSIsInNlcnZlcl92ZXJzaW9uIjoiMS43LjAiLCJzZXJ2ZXJfYnVpbGQiOiIyMzIiLCJmZWF0dXJlX3NldCI6ImJhc2ljIn0.q3lH3kLqjzE1eEHP2vvHVCTwJdjgw1JNQrnTXLUnPc0"}}

4. Have created a Conda Environment and install clearml followed by clearml-init.

NOTE : On the same laptop, the clearml-init works but the clearml-agent init does not work

Please guide me one how I can add workers to the ClearML Server with the clearml-agent init command.

Thanks.

[TalhaUsuf]

I am also facing the same issue, Did you manage to solve it?

[jkhenning]

Hi @vineetsharma14 and @TalhaUsuf,

This is very strange especially since the curl command to 8008 was successful...

Which agent and server version are you using?

[vineetsharma14]

Thanks for your reply.

The version of the ClearML Agent : 1.4.1 (found using the clearml-agent --version command)

The version of the ClearML Server as mentioned in the docker_compose.yaml is Version : 3.6

Please let me know - if you need info from my side.

Thanks for the help ! Really appreciate it.

[ywang-psee]

Hi @vineetsharma14 , i got the same error as you. I resolved by downgrade ClearML Agent version to 1.1.6 : pip install clearml==1.1.6

[jkhenning]

@ywang-psee do you mean the clearml or clearml-agent version? 1.1.6 is very old, does this reproduce with latest versions?

[vineetsharma14]

Any idea on how to fix this.

Thanks for the help !

[NathanClearML]

Hi @vineetsharma14 , can you also try downgrading the clearml-agent version to see if this helps? Just so we can better focus on the issue

[jkhenning]

@vineetsharma14 what does curl http://192.168.0.244:8008/auth.login return?