[RFC] Create a plugin for metric reporting

[chishui]

Is your feature request related to a problem? Please describe.

1. We want to create a plugin to record metrics such as number of click event on an UI element, number of times a UI element is visible to user, the success rate of an API call etc.
2. The metrics are better not be accumulated as we want to keep track of each reported metric.
3. The metric storing solution can be customized so that we can publish them to AWS CloudWatch, or Kinesis, or simply output a log.
4. We don't vend a stat API in OSD as metric data could be stored to all kinds of platform and handled/analyzed/visualized from other platforms.
5. We don't want to store metrics to .kibana index through savedobject by default as it could overextend .kibana index

Describe the solution you'd like

A clear and concise description of what you want to happen.

1. Create a core plugin: MetricsRecorder
2. Other plugins can depend on this plugin and use it to record a metric.
3. The recorded metrics are batched locally and sent to node API to process every 1 minute (can be configured)
4. The plugin vends a function to register a MetricsRecorderFactory so that users can customize the metric storing strategy.
5. The node API will call MetricsRecorderFactory to process the metrics.

Describe alternatives you've considered We've considered using usage_collection plugin, it's a legacy plugin marked as deprecated for quite a long time for some performance issues. Here are some other obvious downsides make it not a good solution:

1. It relies on saved_object. By default, all metrics are stored to .kibana index which as mentioned above is already overextended. Mixing metric data and other user's configuration such as index pattern, dashboard, visualization etc could bring down the performance. Switching to a new storage solution could cause missing data for existing user.
2. People can provide a customized implementation of saved_object, but the interface of it is both complex and not quite relevant to metrics reporting.
3. usage_collection has many logic built to provide the stat() API which is unnecessary for metric collection purpose and add additional burden to customization.

telemetry is a plugin replies on usage_collection and it doesn't store metrics, only fetch.

Additional context

Add any other context or screenshots about the feature request here.