kubeedge
[Feature] KubeEdge SIG AI: Benchmarks for Edge-cloud Collaborative Lifelong Learning
Background: The KubeEdge SIG AI is chartered to facilitate Edge AI applications with KubeEdge. An overview of SIG AI activities can be found in this charter.
KubeEdge-Sedna has released the first edge-cloud collaborative lifelong learning on June 2021, together with a hands-on example and a free playground on Katacoda. The sceme learn coming tasks from the edge based on previously learned tasks on cloud knowledgebase.
Why is this project needed: According to the survey on edge ai landing challenges, the top 1 advice goes to "providing public datasets, pre-processing and baseline codes to build benchmarks". This advice gets 82.18%, 92.98%, 87.10% and 86.67% votes among all paticipants, the industial, acadamic and student community on edge ai, respectively.
Our Effort starts from the edge-cloud collaborative benchmarks. This project focuses on measuring and validating the desired behaviors for an epoch-making Edge AI scheme, i.e., Edge-cloud Collaborative Lifelong Learning.
What contents are to be added/modified: This project will help all Edge AI application developers to validate and select the best-matched algorithm of lifelong learning. Parts of the effort are developing test cases on the existing scheme of Edge-cloud Collaborative Lifelong Learning on KubeEdge-Sedna, including interfaces for benchmark specification, algorithms like preprocessing and metrics, and even baselines.
Other information: Recommended Skills: TensorFlow/Pytorch, Python Sedna lifelong-learning introduction Sedna guide Sedna lifelong-learning proposal How to contribute Sedna
Related issue for reference (if any): #58 Add an example of implementation Federated Learning to ReID #96 Edge AI Benchmark review #118 Edge AI Benchmark: Consolidate/Prioritize metrics #119 Edge AI Benchmark: add step by step instructions for users to benchmark their works #120 Edge AI Benchmark: add leaderboard/forum #274 KubeEdge SIG AI: Benchmarks for Edge-cloud Joint Inference
Several breath-taking Edge-AI scenarios have been prepared for benchmarking: looking forward to seeing your codes involved in real-world robots, outer-space satellites, and industrial production lines!
KubeEdge has launched a new sub-project KubeEdge-Ianvs for distributed collaborative AI benchmarking to handle this issue.
@iszhyang owns a great osop project working on a simulator for this issue! Pls refer to:
- Proposal Pull Request: https://github.com/kubeedge/ianvs/pull/35
- Code Pull Request: https://github.com/kubeedge/ianvs/pull/39
- Demo Video（in Chinese）on KEAW'22: https://www.bilibili.com/video/BV1nD4y1Y7cJ
A related issue on cloud robotics is available at KubeEdge-Ianvs: https://github.com/kubeedge/ianvs/issues/48