kafka-tutorials icon indicating copy to clipboard operation
kafka-tutorials copied to clipboard

Standardizing and Cleansing IoT data before sending downstream

Open ybyzek opened this issue 3 years ago • 0 comments

Describe the use case

There is already an IoT recipe that talks about IoT from the perspective of anomaly detection https://confluentinc.github.io/ksqldb-recipes/anomaly-detection/internet-of-things/ . In contrast, this recipe describes a use case for something that has to happen as a precursor, which is standardizing and cleansing the incoming IoT device data (before it's processed for anomaly detection). Pull raw events, join data between events and sensors, normalize identifiers, assign an event UUID, convert timestamps, assign the key as the device id, etc.

Provide the ksqlDB application

See src (Confluent-internal only)

ybyzek avatar Jan 11 '22 17:01 ybyzek