Send all calls [Encrypted/Not covered] to status server for Stats

[MaxwellDPS]

How hard would it be to have the program send all calls, and events to the status server for analytics and statics?

[kd4e]

It would be good to be able to program an alert for a significant increase in encrypted calls,

just the stats, within a time period (a few hours, days, month).

On 5/7/21 11:36 PM, Max Watermolen wrote:

How hard would it be to have the program send all calls, and events to the status server for analytics and statics?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/462, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARVGGHPDMSEQLPRHM2BU46TTMSWU5ANCNFSM44MEDN6Q.

[MaxwellDPS]

Exactly what I was thinking! Some sort of analysis of the types, and units of all calls. I edited the status server to send to splunk to do analysis on. That could be ported to anything really

Units active

Calls by type

Here is the index.js if youre curious

var express = require('express');

var app = express(),
    http = require('http'),
    server = http.createServer(app);

const WebSocket = require('ws');

const serverWss = new WebSocket.Server({
    noServer: true
});

const url = require('url');


var bodyParser = require('body-parser');
var srv = null;

server.on('upgrade', (request, socket, head) => {
    const pathname = url.parse(request.url).pathname;

    if (pathname === '/server') {
        console.log("Upgrading Server Connection");
        serverWss.handleUpgrade(request, socket, head, (ws) => {
            serverWss.emit('connection', ws);
        });
    } else {
        socket.destroy();
    }
});

app.use(bodyParser());

function heartbeat() {
    this.isAlive = true;
}

function processMessage(call, type){
    const https = require('https')

    const data = JSON.stringify({
        event: {
            data: call,
            type: type
        }

    })
    
    const options = {
        hostname: '<SPLUNKHOST>',
        port: 8088,
        path: '/services/collector/event',
        method: 'POST',
        headers: {
            'Authorization': 'Splunk <SPLUNKTOKEN>'
        }
    }
    
    const req = https.request(options, res => {
       
    
        res.on('data', d => {
        process.stdout.write(d)
        })
    })
    
    req.on('error', error => {
        console.error(error)
    })
    
    req.write(data)
    req.end()
    console.log("posted " + type)
}

serverWss.on('connection', function connection(ws, req) {
    ws.isAlive = true;
    ws.on('pong', heartbeat);
    console.log((new Date()) + ' WebSocket Connection accepted.');
    ws.on('message', function incoming(message) {
        try {            
            var data = JSON.parse(message);
        } catch (err) {
            console.log("JSON Parsing Error: " + err);
        }
        if (data.type == 'calls_active') {
            for (var i = 0; i < data.calls.length; i++) { 
                dataX = data.calls[i];
                for (var i = 0; i < dataX.sourceList.length; i++){
                    processMessage(dataX.sourceList[i], 'unit')
                }
                processMessage(dataX, 'call')
            }
        }
    });
    ws.on('close', function(reasonCode, description) {
        srv = null;
    });

});


server.listen(3010);
module.exports = server;

[MaxwellDPS]

@kd4e Looks like if you comment line 631 in trunk-recorder/main.cc I it will log properly. but It will cause all encrypted transmissions to be saved. so with a little python Im sure that can be solved to not keep those

https://github.com/robotastic/trunk-recorder/blob/1fadb220bbb208016440d34da607941ae57b29ab/trunk-recorder/main.cc#L631

[MaxwellDPS]

@robotastic is there an easy way to accomplish this without saving encrypted audio?

[dotsam]

I've been running a fork where all calls are sent to the stats socket for a while, by commenting out this conditional that checks for a recording state https://github.com/robotastic/trunk-recorder/blob/1fadb220bbb208016440d34da607941ae57b29ab/trunk-recorder/uploaders/stat_socket.cc#L185-L187

[MaxwellDPS]

@dotsam Im sorry I am a bit lost lol, Do you mind sending an example or fork :)

[MaxwellDPS]

@dotsam the following doesnt seem to work

// if (call->get_state() == recording) { 
   node.push_back(std::make_pair("", call->get_stats())); 
 //} 

[dotsam]

I haven't rebased in a while (over a year), and I know there were some big changes around how "calls" were handled (I believe it's now more of a "transmission" than a "call" that's tracked) so it's possible this is no longer sufficient to get everything into the stats socket.

[Dygear]

Ah, that was my change that caused this. I skipped the whole thing on the E flag to block encrpyted talk groups from capturing a recorder that it can't do anything about while I still wanted to have the label for it show up in the console. As it shows up in the console, I think that might be a good area to start to look to see if you can wedge logging code into there for the status server -- Or move the status server code to around that same area so it's logged while still skipping the (going to fail) encrypted recorder audio capture.

[MaxwellDPS]

@Dygear thank you for providing some clarity, would you be able to provide a code sample of the actual logging mechanism.. I'm afraid my c is a little rusty lol

[MaxwellDPS]

Ok, so after some testing I found that we can send the call to the stats server via stats.send_call_start(call);

So if you put that under line 630 like so, it Fixes the issue. I summited pull request #469

 if (call->get_encrypted() == true || (talkgroup && (talkgroup->mode == 'E'))) {
    if (sys->get_hideEncrypted() == false) {
      BOOST_LOG_TRIVIAL(info) << "[" << sys->get_short_name() << "]\tTG: " << call->get_talkgroup_display() << "\tFreq: " << FormatFreq(call->get_freq()) << "\t\u001b[31mNot Recording: ENCRYPTED\u001b[0m ";
    }
    stats.send_call_start(call);
    return false;
  }

[MaxwellDPS]

EDIT: My bad I used the wrong binary, that doesnt work :(

[robotastic]

I really like the original idea of logging system activity and then doing analytics on it. I am going to work on getting the Plugin System working and this could be a good way to flexibly send the Call Info to different places. It looks like a key feature to build in, is making sure all calls get passed through here, even if they don't get uploaded to OpenMHz/Broadcastify.

[MaxwellDPS]

@robotastic That would be excllent! LMK if there is anything I can do to assist

[kcwebby]

...snipped...

const options = {
    hostname: '<SPLUNKHOST>',
    port: 8088,
    path: '/services/collector/event',
    method: 'POST',
    headers: {
        'Authorization': 'Splunk <SPLUNKTOKEN>'
    }
}
...snipped...

@MaxwellDPS I'm extremely interested in how you are sending this to Splunk. I'd very much like to do the same, but I'm using graylog.. I can't find an input method that can accept the data straight from a websocket. What kind of collector configuration in splunk that you are posting this to? Like I said, I'm trying to duplicate with graylog, but I cannot get the status-server to connect to the input process. Thanks in advance.

[MaxwellDPS]

@kcwebby This is done with Splunks HEC's or HTTP Event collectors. All I am doing is doing a HTTP Post with the json to splunk. In theory if graylog doesnt have a webhook you could post this json to any python app or other code and have that convert to a better format.

Trunk Recorder -(WS)-> index.js -(HTTP/POST)-> Splunk Webhook

[kcwebby]

@MaxwellDPS -- Forgive my ignorance, I'm not a mainstream coder -- My import option right now to to send it as raw/plaintext, which means I need to modify your index.js to send to http, instead of https. Right now, I'm getting what I believe to be encrypted log entries in my Graylog, but I do see it there. Could you help me out? I've forked and created a pull request with what I think to be the proper code adaptations, but zero clue how to change to http from https. https://github.com/kcwebby/trunk-recorder-status-server/pull/1

[MaxwellDPS]

@kcwebby Sorry for the late response. I beleve you could modify the following lines like so for an non-TLS HTTP request set const https = require('https') to const http = require('http') set const req = https.request(options, res => { to const req = http.request(options, res => {

Full example

function processMessage(call, type){
    const http = require('http')

    const data = JSON.stringify({
        event: {
            data: call,
            type: type
        }

    })
    
    const options = {
        hostname: '192.168.1.231',
        port: 8088,
        path: '/services/collector/event',
        method: 'POST',
        headers: {
            'Authorization': 'Splunk <>'
        }
    }
    
    const req = http.request(options, res => {    
        res.on('data', d => {
        process.stdout.write(d)
        })
    })
    
    req.on('error', error => {
        console.error(error)
    })
    
    req.write(data)
    req.end()
    console.log("posted " + type)
}