Feat/websocket service

[cemathey]

tl;dr this adds an optional service that allows API consumers to use web sockets to have log events pushed to them rather than having to constantly poll one of the HTTP endpoints and re-process old logs looking for new ones.

The fine people over at The Circle have been using this to support one of their tools and have been testing one version or another of this for a few months.

It works similarly to the other services, it has a supervisord section in the config, has a standard UserConfig setup which is manageable through the UI and the service is toggled on the Settings page.

It does launch a separate webserver (daphne) which is used to handle the websockets but luckily we're able to use the same ports externally, the connection will auto negotiate when it recognizes a web socket connection with nginx in the frontend containers.

Notes I drafted to add to the Wiki when it's released:

Overview

The log_stream is a Redis stream that stores logs from the game server in sequential order on a transient basis (they are not persisted to the database and are cleared on service startup) to support pushing new logs to external tools through a websocket endpoint.

Configuration

{
  "enabled": false,
  "stream_size": 1000,
  "startup_since_mins": 2,
  "refresh_frequency_sec": 1,
  "refresh_since_mins": 2
}

stream_size: The number of logs the stream will retain before discarding the oldest logs.

startup_since_mins: The number of minutes of logs to request from the game service when the service starts up

refresh_frequency_sec: The poll rate for asking for new logs from the game server

refresh_since_mins The number of minutes of logs to request from the game service each loop

Permissions

For new installs these are added to the owner and admin group.

Existing installs can add these permissions to users or groups:

1. api.can_view_log_stream_config
2. api.can_change_log_stream_config

Service

The log_stream service will start by default but will only actually do anything if enabled is set to true in its config.

Each start up the redis stream is purged and logs (if any are available) are pulled from the game server.

Stream IDs

In a Redis stream the IDs must be unique and sequentially increasing.

We use the timestamp of a log entry and a 0 indexed incrementing suffix because more than one log can occur at the same timestamp, e.g. 1711657986-0 and 1711657986-1.

Connecting / Authentication

To connect you must make a web socket connection to your CRCON at your normal RCONWEB_PORT or RCONWEB_PORT_HTTPS with the /ws/logs endpoint.

For example: ws://localhost:8010/ws/logs

You must include a valid API key for an account with the appropriate permisisons, there is no username/password based authentication.

Request Formats

After successfully connecting you must send a JSON message with your criteria before you will receive any logs.

You can specify a past stream ID (last_seen_id) and/or filter by action type (actions) (any valid type as defined in rcon.types.AllLogTypes).

If you don't care about past logs, simply pass null for last_seen_id, or if you don't care about either you can pass {}.

No filter and only new logs:

{}

Older logs:

{ "last_seen_id": null }

Filtered logs:

{ "actions": ["KILL", "TEAM KILL"] }

Response Formats

Data is return as JSON and is in the following formats:

Service disabled:

{
    "error": "Log stream is not enabled in your config",
    "last_seen_id": null,
    "logs": []
}

If more than 25 logs would be returned, they're batched 25 logs per response and multiple responses are sent to avoid any potential issues with payload size.

Logs:

{
    "last_seen_id": "1711657986-1",
    "logs": [
        {
            "id": "1711657986-0",
            "log": {
                "version": 1,
                "timestamp_ms": 1711657986000,
                "relative_time_ms": -188.916,
                "raw": "[449 ms (1711657986)] KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "line_without_time": "KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "action": "KILL",
                "player": "maxintexas13",
                "steam_id_64_1": "76561199195150101",
                "player2": "Twisted",
                "steam_id_64_2": "76561199096984835",
                "weapon": "KARABINER 98K",
                "message": "maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "sub_content": null
            }
        },
        {
            "id": "1711657986-1",
            "log": {
                "version": 1,
                "timestamp_ms": 1711657986000,
                "relative_time_ms": -188.916,
                "raw": "[340 ms (1711657986)] TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "line_without_time": "TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "action": "TEAM KILL",
                "player": "Untreated HSV",
                "steam_id_64_1": "76561198039482575",
                "player2": "zeto61",
                "steam_id_64_2": "76561198178304393",
                "weapon": "MK2 GRENADE",
                "message": "Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "sub_content": null
            }
        }
    ],
    "error": null
}