PROTOTYPE: Implementation of lightweight events
PROTOTYPE WORK:
Lightweight events are custom high-volume, high-throughput events sent to Microsoft's OneDS (One Collector) service account associated with PlayFab user's entity ID. It is expected that these events can be quickly emitted at a high frequency from different threads of a customer app (i.e. non-blocking thread-safe emit event calls), batched up according to specific settings, and sent out (whether when a batch is full or an allowed timeframe to assemble a batch ran out) to the OneDS service from a background thread. Sending events to OneDS service requires specific authentication info (tokens, IDs) that must be obtained first in runtime by calling PlayFab's "/Event/GetTelemetryIngestionConfig" API.
The names of classes are specifically chosen to be consistent with C++ implementation.
The code in this PR includes the following major components:
- Classes in <SDK>/source/external/aria is a bare minimum custom dependency carefully taken from much larger Aria SDK to support Bond serialization and Common Schema 3.0 composition required by OneDS service
- OneDSSysHttpPlugin is an HTTP plugin specialized to communicate with OneDS server
- OneDSEventsAPI is a OneDS-equivalent of PlayFabEventsAPI: it provides an API method to send an already assembled batch of events to OneDS server. In addition, this class includes an API method to get OneDS authentication info from PlayFab server (GetTelemetryIngestionConfigAsync)
- OneDSEventsModels contains data models required for OneDSEventsAPI
- PlayFabEvent is a class that represents a custom event (such as a lightweight event, for example. There can be other types of custom events in the future)
- PlayFabEventAPI is an API that allows to emit custom events (such as lightweight events)
- PlayFabEventRouter is a class that routes custom events to different event processing pipelines, depending on the type of an event and other conditions
- Classes in <SDK>/source/Pipeline contain the implementation of an event processing pipeline that is largely based on the pipeline implementation from the Internal C# SDK (used in PlayFab Thunderhead services). We would like to keep these implementations as close as possible (and have only one, eventually). A pipeline consists of several buffers (event buffer, batch buffer) and stages (batching stage, sending stage). Each stage reads an item from one buffer, processes it, and puts it to another buffer. The goal is to eliminate performance bottlenecks associated with sending batches over HTTP and, to some degree, batching. The pipeline code was reasonably tested in Thunderhead environment.
- OneDSEventPipelineSettings contains parameters that regulate main technical characteristics of an event pipeline (size of buffers, size of event batches, max wait interval before a batch needs to be sent out, etc).
- <SDK>/source/logger contains classes that provide a logger interface (and the default debug logger) used in event pipelines
- Test for OneDSEventsAPI that sends an already prepared batch of events to OneDS server
- Test for PlayFabEventAPI that emits many lightweight events and uses a OneDS event pipeline to batch them in background, and send those batches to OneDS server