aiops-engine-for-skywalking
aiops-engine-for-skywalking copied to clipboard
This is an incubating repository of the Apache SkyWalking AIOps Engine
This commit contains ray-based code that can consume data from a MQ.
Background: Drain log parsing works best on ingesting only log content - meaning we trim the rest with some simple Regex or rule. Slicing the contents accurately from ` Dec...
Once we have the log clusters learnt by Drain,, we can enable anomaly detection without needing algorithms but simply plot them in time. The idea is simple: if a type...
The following paper was recently published and claims to outperform Drain with some improvements. Mainly the classification of logs and templates based on propositional logic. Details 1) Proposition 1: the...
A sudden thought, We can avoid masking parts before actual message using this heuristic. True effect needs evaluation. And this is not universal, so we only provide it as a...
Now, @Liangshumin has a prototype to use cache look-up to speed up Drain significantly. I changed the lookup to after masking since - I intend to ingest raw log, and...
- [x] A fundamental Plugin framework. - [x] An gRPC data ingestor provider, I will provide a working example for [log](https://github.com/SkyAPM/aiops-engine-for-skywalking/issues/6), it can be used as an example to write...
AIOps engine will receive a large amount of log data from SkyWalking, and we decided to utilize a Redis stream as the buffer before stream processing. One noticeable issue is...
After the initial evaluation phase, we would start our log analysis feature by implementing the [Drain](http://jiemingzhu.github.io/pub/pjhe_icws2017.pdf) method. The goal of this algorithm is to ingest a raw log record stream...
The goals of our project are: - [ ] A gRPC data source for metrics (Actually I'm not quite sure with this, I'd refer to the implementation of the gRPC...