Tansu

Tansu is a distributed key value store designed to maintain configuration and other data that must be highly available. It uses the Raft Consensus algorithm for leadership election and distribution of state amongst its members. Node discovery is via mDNS and will automatically form a mesh of nodes sharing the same environment.

Features

Key Value Store

Tansu has a REST interface to set, get or delete the value represented by a key. It also provides a HTTP Server Sent Event Stream of changes to the store.

Check And Set

Tansu provides REST interface for simple Check And Set (CAS) operations.

Locks

Tansu provides test and set operations that can be used to operate locks through a simple REST based HTTP Server Sent Event Stream interface.

Quick Start

To start a 5 node Tansu cluster using Docker:

for i in {1..5}; do
    docker run \
        --name tansu-$(printf %03d $i) \
        -d shortishly/tansu;
done

The following examples use a random Tansu node:

RANDOM_IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} tansu-$(printf %03d $[1 + $[RANDOM % 5]]))

You can use the same ${RANDOM_IP} for each example, or pick a new one each time. Tansu will automatically proxy any requests that must be handled to the leader if necessary (typically, locks, CAS and writes are handled by the leader) or can be handled by a follower (reads are handled directly by followers).

Key Value Store

Stream changes to any key below "hello":

curl -i -s "http://${RANDOM_IP}/api/keys/hello?stream=true&children=true"

Note that you can create streams for keys that do not currently exist in the store. Once a value has been assigned to the key the stream will issue change notifications. You can also listen for changes to any key contained under the sub hierarchy by adding children=true to the query.

Set

In another shell assign the value "world" to the key "hello":

curl -X PUT -i -s http://${RANDOM_IP}/api/keys/hello -d value=world

The stream will now contain a create notification:

id: 1
event: create
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"text/plain","created":1,"parent":"/","updated":1}},"value":"world"}

Or a key that is below "hello":

curl -X PUT -i -s http://${RANDOM_IP}/api/keys/hello/joe -d value=mike

The stream will now contain a create notification:

id: 2
event: create
data: {"category":"user","key":"/hello/joe","metadata":{"tansu":{"content_type":"text/plain","created":2,"parent":"/hello","updated":2}},"value":"mike"}

Or with a content type:

curl -X PUT -H "Content-Type: application/json" -i http://${RANDOM_IP}/api/keys/hello --data-binary '{"stuff": true}'

With an update in the stream:

id: 3
event: set
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"application/json","created":1,"parent":"/","updated":3}},"previous":"world","value":{"stuff":true}}

Get

The current value of "hello":

curl -i -s http://${RANDOM_IP}/api/keys/hello

{"stuff": true}

Delete

Ask a random member of the cluster to delete the key "hello":

curl -i -X DELETE http://${RANDOM_IP}/api/keys/hello

The stream now contains a delete notification:

id: 5
event: delete
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"application/json","created":1,"parent":"/","updated":5}},"value":{"stuff":true}}

TTL

A value can also be given a time to live by also supplying a TTL header:

curl -X PUT -H "Content-Type: application/json" -H "ttl: 10" -i http://${RANDOM_IP}/api/keys/hello --data-binary '{"ephemeral": true}'

The event stream will contain details of the create together with a TTL attribute:

id: 6
event: create
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"application/json","created":6,"parent":"/","ttl":10,"updated":6}},"value":{"ephemeral":true}}

Ten seconds later when the key is removed:

id: 7
event: delete
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"application/json","created":6,"parent":"/","ttl":0,"updated":7}},"value":{"ephemeral":true}}

Test and Set

Set the value of /hello to be jack only if that key does not already exist:

curl -X PUT -i http://${RANDOM_IP}/api/keys/hello?prevExist=false -d value=jack

The stream identifies test and set changes with type=cas:

id: 8
event: set
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"text/plain","created":8,"parent":"/","type":"cas","updated":8}},"value":"jack"}

Set the value of /hello to be quentin only if its current value is jack:

curl -X PUT -i http://${RANDOM_IP}/api/keys/hello?prevValue=jack -d value=quentin

id: 9
event: set
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"text/plain","created":8,"parent":"/","type":"cas","updated":9}},"previous":"jack","value":"quentin"}

Delete the value of /hello only if its current value is quentin:

curl -X DELETE -i http://${RANDOM_IP}/api/keys/hello?prevValue=quentin

id: 10
event: delete
data: {"category":"user","key":"/hello","metadata":{"tansu":{"content_type":"text/plain","created":8,"parent":"/","updated":10}},"value":"quentin"}

Locks

Locks are obtained by issuing a HTTP GET on /api/locks/ followed by the name of the lock. The response is a Server Sent Event stream that will indicate the status the lock to the caller. The lock holder will retain the lock until the connection is dropped (either by the client or sever). The free lock is then automatically granted to a waiting connection.

In several different shells simultaneously request a lock on "abc":

curl -i -s http://${RANDOM_IP}/api/locks/abc

curl -i -s http://${RANDOM_IP}/api/locks/abc

curl -i -s http://${RANDOM_IP}/api/locks/abc

One shell is granted the lock, with the remaining shells waiting their turn. Drop the lock by hitting ^C on the holder, the lock is then allocated to another waiting shell.

Leadership Election

Tansu provides cluster information via the /api/info resource as follows, picking a random node:

curl -s http://${RANDOM_IP}/api/info|python -m json.tool

Each node may be in follower or candidate state, with only one node in the leader role:

{
    "applications": {
        "any": "rolling",
        "asn1": "4.0.2",
        "cowboy": "2.0.0-pre.2",
        "cowlib": "1.3.0",
        "crown": "0.0.1",
        "crypto": "3.6.3",
        "envy": "0.0.1",
        "gproc": "git",
        "gun": "1.0.0-pre.1",
        "inets": "6.2.2",
        "jsx": "2.8.0",
        "kernel": "4.2",
        "mdns": "0.4.1",
        "mnesia": "4.13.4",
        "public_key": "1.1.1",
        "ranch": "1.1.0",
        "recon": "2.2.1",
        "rfc4122": "0.0.3",
        "sasl": "2.7",
        "shelly": "0.1.0",
        "ssh": "4.2.2",
        "ssl": "7.3.1",
        "stdlib": "2.8",
        "tansu": "0.13.0"
    },
    "consensus": {
        "cluster": "fadbf747-f700-4d87-b986-73c2a1de18e4",
        "commit_index": 13668,
        "connections": {
            "1c64ca8a-303b-43f7-914f-09e3b680f9ed": {
                "host": "172.17.0.2",
                "port": 80
            },
            "41fc20ae-70be-4e9d-a40d-a8164b165283": {
                "host": "172.17.0.7",
                "port": 80
            },
            "6343a50d-acc6-4f62-866d-b5a7dcde5d04": {
                "host": "172.17.0.5",
                "port": 80
            },
            "e8d25d82-5f07-4598-bb0c-6074793ee111": {
                "host": "172.17.0.3",
                "port": 80
            },
            "e9057f76-24b5-4f4f-b81c-96d555798ef4": {
                "host": "172.17.0.4",
                "port": 80
            }
        },
        "env": "dev",
        "id": "f327cc37-114f-4237-942b-972199e364a1",
        "last_applied": 13668,
        "leader": {
            "commit_index": 13668,
            "id": "e8d25d82-5f07-4598-bb0c-6074793ee111"
        },
        "role": "follower",
        "term": 877
    },
    "version": {
        "major": 0,
        "minor": 13,
        "patch": 0
    }
}

This section optionally uses jq to parse some of the JSON output from Tansu. To install use:

dnf install -y jq

The following script iterates over the members of the Tansu cluster outputting the role of each member:

for i in {1..5};
do
NAME=tansu-$(printf %03d $i)
IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} ${NAME})
ROLE=$(curl -m 1 -s http://${IP}/api/info|jq .consensus.role)
echo ${NAME} ${ROLE}
done

As an example:

tansu-001 "follower"
tansu-002 "leader"
tansu-003 "follower"
tansu-004 "follower"
tansu-005 "follower"

Pause the leader (replace tansu-002 with your leader):

docker pause tansu-002

Check that one of the other nodes has been established as the leader by repeating the curl of /api/info on the remaining nodes:

for i in {1..5};
do
NAME=tansu-$(printf %03d $i)
IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} ${NAME})
ROLE=$(curl -m 1 -s http://${IP}/api/info|jq .consensus.role)
echo ${NAME} ${ROLE}
done

As an example, tansu-002 is now paused:

tansu-001 "leader"
tansu-002
tansu-003 "follower"
tansu-004 "follower"
tansu-005 "follower"

Fire some updates into one of the remaining nodes (change tansu-003 to a running node):

IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} tansu-003)
for i in {0..100};
do
    curl -X PUT -s http://${IP}/api/keys/pqr -d value=$i;
done

Tansu will create a snapshot of its current state every so often, and uses this snapshot when members join (or rejoin) the cluster together with any log entries subsequent to that snapshot.

Unpause the diposed former leader:

docker unpause tansu-002

Check that tansu-002 has rejoined the cluster and is now in the follower role:

for i in {1..5};
do
NAME=tansu-$(printf %03d $i)
IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} ${NAME})
ROLE=$(curl -m 1 -s http://${IP}/api/info|jq .consensus.role)
echo ${NAME} ${ROLE};
done

The node tansu-002 is now in the follower role:

tansu-001 "leader"
tansu-002 "follower"
tansu-003 "follower"
tansu-004 "follower"
tansu-005 "follower"

Ask the unpaused node for the value of pqr:

IP=$(docker inspect --format={{.NetworkSettings.IPAddress}} tansu-002)
curl -i http://${IP}/api/keys/pqr

The node will have same value as the remainder of the cluster:

100

Configuration

Tansu uses the following configuration environment.