Port unavailable error when running in colab

[cheburakshu]

System Info

Colab Pro T4

Information

• [ ] Docker
• [X] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

Problem: The text embeddings router always fails to start with error even though the port is not in use. Error: failed to create HTTP listener: error creating server listener: Address already in use (os error 98)

./usr/local/bin/text-embeddings-router --port 8081

2024-04-24T14:49:07.908654Z  INFO text_embeddings_router: router/src/main.rs:140: Args { model_id: "the*****/***-*ase", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, hf_api_token: None, hostname: "4a1c28c1c2e8", port: 8081, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: None, payload_limit: 2000000, api_key: None, json_output: false, otlp_endpoint: None, cors_allow_origin: None }
2024-04-24T14:49:07.908756Z  INFO hf_hub: /root/.cargo/git/checkouts/hf-hub-1aadb4c6e2cbe1ba/b167f69/src/lib.rs:55: Token file not found "/root/.cache/huggingface/token"    
2024-04-24T14:49:08.010515Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:20: Starting download
2024-04-24T14:49:08.010609Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:37: Model artifacts downloaded in 97.543µs
2024-04-24T14:49:08.023091Z  INFO text_embeddings_router: router/src/lib.rs:169: Maximum number of tokens per request: 512
2024-04-24T14:49:08.023189Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:23: Starting 1 tokenization workers
2024-04-24T14:49:08.026630Z  INFO text_embeddings_router: router/src/lib.rs:194: Starting model backend
2024-04-24T14:49:08.254455Z  INFO text_embeddings_backend_candle: backends/candle/src/lib.rs:152: Starting FlashBert model on Cuda(CudaDevice(DeviceId(1)))
2024-04-24T14:49:09.338769Z  WARN text_embeddings_router: router/src/lib.rs:263: Invalid hostname, defaulting to 0.0.0.0
Error: failed to create HTTP listener: error creating server listener: Address already in use (os error 98)

Please use this notebook to test https://colab.research.google.com/drive/1PsqVNPRiTsVp0bHA_0dByZ-1uiFUl_lN?usp=sharing

FYI: I have run databases like neo4j, postgres inside colab and faced no issues.

Expected behavior

The server starts normally when the port is unused.