kismet icon indicating copy to clipboard operation
kismet copied to clipboard

Push live data to No Sql database

Open tonybounty opened this issue 5 years ago • 6 comments

It's possible to write in live datas/alerts/devices... in txt file or to stdout instead of sqlite file ? I have to put all results in No SQL DB, so I need to parse data from kismet. Sqllite isn't appropriate to do that.

tonybounty avatar Nov 30 '19 22:11 tonybounty

The kismetdb log contains full JSON records; you can put them through whatever you want (nosql, elk, whatever you want to format it as), so you can either process the existing data out of the kismet log into whatever you want - using for example the python-kismetdb code or the JSON export code, or you can talk to the kismet rest ui (docs are all on the website). Either one gives you full JSON records; either from an existing log, or realtime.

On Sat, Nov 30, 2019 at 5:36 PM tonybounty [email protected] wrote:

It's possible to write in live datas/alerts/devices... in txt file or to stdout instead of sqlite file ? I have to put all results in No SQL DB, so I need to parse data from kismet. Sqllite isn't appropriate to do that.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kismetwireless/kismet/issues/193?email_source=notifications&email_token=AFKJYY3RCBHV4HSWDQVSXU3QWLTHFA5CNFSM4JTIUG7KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H5A4YYA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFKJYY6P6OGTX4DSZX4L4DTQWLTHFANCNFSM4JTIUG7A .

kismetwireless avatar Nov 30 '19 22:11 kismetwireless

Do you have more concrete example ? I need do push every new data from kismet to ELK in realtime to update my "live dashboard". However, when I launch Kismet it give me a SQLite 3 file. It's very complicated to parse SQL on every new data because it's a structured file, and this consume lot of resource to request on every new data. The more convenient way to parse log is to have a simple text log file (in JSON for eg, line by line), then ELK/Prometheus... or other log aggregator can detect and read each new line written.

Can kismet do that ?

tonybounty avatar Dec 01 '19 15:12 tonybounty

Have a look at kismet_rest on github, I think that is what you are looking for

Sent from my iPhone

On 1 Dec 2019, at 15:47, tonybounty [email protected] wrote:

 Do you have more concrete example ? I need do push every new data from kismet to ELK in realtime, to update my "live dashboard". However, when I lunch Kismet it give me a SQLite 3 file. It's very complicated to parse SQL on every new data because it's a structured file, and this consume lot of resource to request on every new data. The more convenient way to parse log is to have a simple log file text (in JSON for eg, line by line), then ELK/Prometheus... or other log aggregator can detect and read each new line written.

Can kismet do that ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

taospartan avatar Dec 01 '19 16:12 taospartan

So if I understand, if I want to get all the alerts in realtime and push it in ELK, I need to create a bash script with a while loop that content :

  • curl request to http://log:[email protected]:2501/alerts/last-time/$LAST_TIME/alerts.json
  • push the JSON result to ELK (log stash)
  • update $LAST_TIME+1
  • sleep 1 sec.

Am I good ?

tonybounty avatar Dec 01 '19 17:12 tonybounty

I’ve done that with python and the kismet_rest package but bash will do it to

Sent from my iPhone

On 1 Dec 2019, at 17:01, tonybounty [email protected] wrote:

 So if I understand, if I want to get all the alerts in realtime and push it in ELK, I need to create a bash script with a while loop that content a

curl request to http://log:[email protected]:2501/alerts/last-time/$LAST_TIME/alerts.json push the JSON result to ELK (log stash) update $LAST_TIME+1 sleep 1 sec. Am I good ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

taospartan avatar Dec 01 '19 17:12 taospartan

I’ve done that with python and the kismet_rest package but bash will do it to Sent from my iPhone On 1 Dec 2019, at 17:01, tonybounty @.> wrote:  So if I understand, if I want to get all the alerts in realtime and push it in ELK, I need to create a bash script with a while loop that content a curl request to @.:2501/alerts/last-time/$LAST_TIME/alerts.json push the JSON result to ELK (log stash) update $LAST_TIME+1 sleep 1 sec. Am I good ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Would you mind sharing? I'd like to see how you did it as I need a MongoDB version.

tranzmatt avatar Feb 24 '20 19:02 tranzmatt