cnf-testbed icon indicating copy to clipboard operation
cnf-testbed copied to clipboard

Spec: Test results structure, documentation and automation

Open taylor opened this issue 6 years ago • 4 comments

Structure and documentation for test results

Goal:

  • easy to find test results
  • easy to understand test results
  • allow automation of running and saving selected set of tests

Planned changes

  • Make the test framework (NFVbench) web-based
    • Control through HTTP (REST/json). Maybe make use of Python API
    • Results available through Kibana/Grafana (Elasticsearch)
  • Configure Fluentd to handle data between NFVbench and Elasticsearch
  • Create dashboards for Kibana/Grafana

Tasks:

  • [x] Containerized deployment of Elasticsearch
  • [x] Containerized deployment of Kibana/Grafana
  • Reconfigure NFVbench for CNF Testbed
    • [ ] Use web API for control
    • [x] Push data to Fluentd
  • Set up web-based control of NFVbench
    • [ ] (maybe) create web-page for controlling NFVbench
    • [ ] (alternative) provide tools for easier use of REST API
  • Set up Kibana/Grafana to parse/display test results
    • [x] Set up web interface
    • [x] Create dashboard(s)
  • [ ] Create documentation for how to use

taylor avatar Oct 03 '19 15:10 taylor

Quick ideas:

  • New folder test_reults with subfolders for different tests (probably associated with use cases/examples)
  • Each test folder would have a dated test run subfolder for a test run
  • Each test folder would have a README documenting what is being tested and the result data (eg. maximum receive rate)

Nice to have:

  • summary of results in each test run subfolder
  • link to latest summary of results from test folder

taylor avatar Oct 03 '19 15:10 taylor

Updated task with more details. Started looking at implementation

Not sure if this should be included in spec, or in a separate task

michaelspedersen avatar Dec 05 '19 14:12 michaelspedersen

(WIP) Data sent from NFVbench though Fluentd to Elasticsearch. Visualized using Kibana dashboard

image

michaelspedersen avatar Dec 10 '19 15:12 michaelspedersen

Documentation has been added for the visualization part

michaelspedersen avatar Dec 18 '19 14:12 michaelspedersen