Namespaces not available in replica

[ryands17]

Hey folks,

I have a docker-compose setup as follows:

services:
  db:
    image: ghcr.io/tursodatabase/libsql-server:e853d54
    volumes:
      - db_data:/var/lib/sqld
    ports:
      - 8080:8080
      - 8082:8082
    environment:
      - SQLD_NODE=primary
      - SQLD_HTTP_LISTEN_ADDR=0.0.0.0:8080
      - SQLD_GRPC_LISTEN_ADDR=0.0.0.0:5001
      - RUST_LOG=debug
    command: [ 'sqld', '--db-path', '/var/lib/sqld/primary.sqld', '--admin-listen-addr', '0.0.0.0:8082', '--enable-namespaces' ]
  replica:
    image: ghcr.io/tursodatabase/libsql-server:e853d54
    volumes:
      - db_replica:/var/lib/sqld
    depends_on:
      - db
    ports:
      - '8081:8080'
    environment:
      - SQLD_NODE=replica
      - SQLD_PRIMARY_URL=http://db:5001
      - RUST_LOG=debug
volumes:
  db_data:
  db_replica:

Without namespaces, this setup works fine and everything is replicated to the replica instance. But as soon as I enable namespaces and create a new namespace via the Admin API, it's not available in the replica and only in the primary.

Is there something I'm missing in my compose file?

[pauchiner]

Hi @ryands17, one little question, how you managed to use the namespaces?

I have a docker container with almost the same configuration as you but without the replica enabled, and I created two namespaces with the Admin API.

But when I try connecting to them under the http://localhost:PORT/dev/NAMESPACE it fails.

Im only able to connect to the default namespace with http://localhost:PORT

Update

I tried with turso db shell http://localhost:PORT/dev/NAMESPACE command and it works.

But I can't connect to the namespaces from a external sqlite client, (Dataflare, Drizzle Studio)

[djalmajr]

But I can't connect to the namespaces from a external sqlite client, (Dataflare, Drizzle Studio)

Same here... I would also like to know.

[hatemjaber]

I know you guys were trying to get the namespace working or how to access it etc... I use a vscode extension called rest client but you can adapt this to whatever you use for api calls:

@api=http://localhost:6080
@api_admin=http://localhost:8082
@api_token={{$dotenv VSCODE_DATABASE_AUTHENTICATION}}

POST {{api}}/v2/pipeline
Authorization: Bearer {{api_token}}

{
  "requests": [
    { "type": "execute", "stmt": { "sql": "SELECT 1 as one" } },
    { "type": "close" }
  ]
}

###

POST http://db1.db.sarna.dev:6080
Content-Type: application/json
Authorization: Bearer {{api_token}}

{
  "statements": [
    { "q": "SELECT 1 AS one" }
  ]
}

###

GET {{api}}/health
Authorization: Bearer {{api_token}}

###

GET {{api}}/dump
Authorization: Bearer {{api_token}}

###

GET {{api}}/version
Authorization: Bearer {{api_token}}

###

POST {{api_admin}}/v1/namespaces/db2/create
Content-Type: application/json

{}

###

DELETE {{api_admin}}/v1/namespaces/db2

###

POST {{api_admin}}/v1/namespaces/test/fork/test-db
Content-Type: application/json

{}

here's my docker...

sqld_primary:
  image: ghcr.io/tursodatabase/libsql-server:latest
  platform: linux/amd64
  container_name: sqld_primary
  hostname: sqld_primary
  command: ["/bin/sqld", "--enable-namespaces", "--admin-listen-addr", "0.0.0.0:8082"]
  ports:
    - "0.0.0.0:6080:8080"  # Expose primary on port 8080
    - "0.0.0.0:8082:8082"  # Expose admin on port 8082
  volumes:
    - ./.sqld/primary:/var/lib/sqld  # Persist primary DB on the host
  environment:
    - SQLD_NODE=primary
    - SQLD_HTTP_AUTH=turso:turso

If you run on apple silicon you need to keep the platform: linux/amd64, otherwise you can remove it. the docker compose config should get you up and running, I mount the volume to local disk for easy access, you can put it in a docker volume...

if you want to access the local setup as if you were hitting a separate subdomain for each db, add an entry to /etc/hosts or your hosts file on whatever system with this 127.0.0.1 *.db.sarna.dev. this will allow you to create the namespace you created and hit that db directly. the only commands I found were to create, delete, and fork a namespace which all three work no problems.

I hope this helps you guys figure out your specific issues.

[ryands17]

@hatemjaber When you insert an item in the primary, does the item exist in the replica? I wasn't able to get that working. The replica doesn't have any data

[hatemjaber]

I assume you're referring to the local replica. If you're using javascript/typescript, this is how you can do it with drizzle:

import { drizzle } from 'drizzle-orm/libsql';
import { createClient } from '@libsql/client';

const client = createClient({
    url: file:name-of-db.db,
    syncUrl: http://localhost:6080,
    syncInterval: 1000,
    authToken: token
});
export const db = drizzle(client, { logger: true });

Once you fire it up and migrate, insert data, etc... it shows up in the local version of the db. I set the sync interval to one second, i'm not sure how that will play out with a remote server hosting the instance. I ran into issues when I first tried where I was not seeing the db in my local project, this is what worked for me. Let me know if this helps you, I hope it does!

[hatemjaber]

@ryands17 I can confirm that the server setup and the local instance are both synced up. I created db.db.sarna.dev:8080 and db2.db.sarna.dev:8080 to test whether or not that would work and they both worked with insert, select. I did not bother testing other methods since those both worked. I hope that helps!

[ryands17]

@hatemjaber No I meant the replica in the server itself. Check the dockerfile I posted where I create a namespace in one replica but it fails to register in the replica

[ryands17]

@pauchiner @djalmajr You need to setup a local domain using dnsmasq as shown here and then you can connect to each database using a DB client via the domain.

[hatemjaber]

@ryands17 you can also use the *.db.sarna.dev for local testing by adding that to your /etc/hosts file.

[hugo-abdou]

@hatemjaber thanks for the api docs i was looking for this informations for a long time now and there is no docs for this part thanks lot