ipfs
Open Grant Proposal - IPFS K8s OpenKube Extensions
Open Grant Proposal: IPFS K8s OpenKube Extensions
Name of Project: IPFS K8s OpenKube Extensions & Middleware Support
Proposal Category: devtools-libraries
Proposer: bsampaio78
Technical Sponsor:
Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT, APACHE2, or GPL licenses?: Yes
Project Description
The IPFS ecosystem has flourished in the past years, with several thousand applications and developers using the power of IPFS - however we believe that non-functional requirements such as abstracting connection and usage of the public IPFS resource network has not been addressed and hinders adoption.
This dev-grant proposal, OpenKube K8s IPFS Operator will enable automatic storage orchestration by mounting IPFS from the comfort of K8s dashboard. We believe operators are needed to enable large scale adoption by the wider developer pool more familiar with traditional software deployment practices and for enterprise adoption of IPFS.
Being built upon Kubernetes, this dev-grant brings security, observability, metrics, alarms, costs, and usability many developers are familiar with from systems and solutions they develop, run, and operate in their jobs.
Simply put, this grant deliverable will assist web2 developers in all key aspects regarding configuration, operation, costs and security of IPFS related resources.
Value
The concept of software operators is mainly to automate and replace knowledge of experienced human operators within a given expertise field. This leads to the wider middleware severing the software operator acting as an abstraction layer, connecting to or actively using resources that is relevant to this abstraction layer.
One can view a protocol like IPFS is facilitating functional requirements for developers and organisations, to interact with the protocol itself. However this interaction can be improved and hence enhance overall developer experience and access. This dev-grant proposal, will facilitate the non-functional requirements developers and large scale organisations may find useful.
Key value concepts this this dev-grant proposal will facilitate:
- Installation: Automated installation of the application with the desired state defined in the custom resource.
- Upgrade: Automated and user-triggered upgrade of the application with minimum user interaction.
- Lifecycle Management: Initialisation, backup, and failure recovery for the applications with decision rules and automation.
- Monitoring and Scalability: Monitor and analyse the metrics and alerts for the application. Take automated actions for scaling, scheduling, and rebalancing when necessary.
Risks
- OpenKube depends on three pillars - Kubernetes, Go programming language, and knowledge about IPFS and Filecoin. Whereas the team has large experience with Kubernetes and Go, knowledge about IPFS and Filecoin are missing and represent a high risk with regards to product roadmap and development.
- Protocol Abstraction Layer and open source governance model mitigate IPFS and Filecoin associated risks, shortening Time To Market.
- Need to establish strong partnerships that can help us to tackle IPFS and Filecoin specificities.
Deliverables
This dev-grant proposal is a part of a larger development initiative by the open source team at OpenKube. Below you will find a diagram, visualising the larger development initiative for both IPFS & Filecoin.
OpenKube roadmap consists of the following milestones / plateaus:
- Selected Kubernetes Distro with Protocol Tools
- Use of Protocols via Kubernetes APIs
- Protocol Abstraction Layer
- Optimal Usage of Protocol Resources
- Kubernetes Open Service Broker
Specifically for the IPFS dev-grant we propose the following deliverables, labelled Kubectl IPFS Plugin, where this plugin will be useful for first, anyone using K8s. Then secondly, individuals who will use the wider OpenKube open-source as described above.
Development Roadmap
Milestone 1 - Selected Kubernetes Distro with Protocol Tools
Using MicroK8s as baseline, OpenKube distribution will contain best existing tools for the IPFS & Filecoin ecosystems. OpenKube will be available from the most popular package management systems for several operating systems.
| Deliverable | Hours Est. |
|---|---|
| Protocol Tools | 160 |
| Selected Kubernetes Distro | 24 |
| Project Management | 40 |
| Total | 224 |
In total, we estimate around 224 hours to deliver milestone-1, the workload will be spread amongst two developers and we will be able to deliver within 1.5 months.
Milestone 2 - Kubectl Plugins
kubectl is a command-line tool to control the Kubernetes cluster manager, familiar to all Kubernetes users, and it can be extended via programs called plugins. In addition to core kubectl commands, several plugins are available, such as cert-manager. A kubectl plugin is designed to support a custom resource and implements several sub-commands. In this work package, the main deliverables are IPFS and Filecoin kubectl plugins.
| Deliverable | Hours Est. |
|---|---|
| Kubectl IPFS Plugin | 480 |
| Kubectl FIL Plugin | 480 |
| Project Management | 120 |
| Total | 1080 |
In total, we estimate around 1080 hours to deliver milestone-2, the workload will be spread amongst two developers and we will be able to deliver within 3 months.
Milestone 3 - Protocol Abstraction Layer
An abstraction layer shields Kubernetes users from protocol particularities. Once the user masters concepts and operations implemented by such a layer, it is much easier to start using new protocols, as well as finding the best protocols addressing his needs. Once the unified data model is defined, existing / new Kubernetes tools will be customised / created to deliver a great user experience.
| Deliverable | Hours Est. |
|---|---|
| Unified Data Model | 80 |
| Unified Dashboards | 80 |
| Kubernetes Tools | 160 |
| Project Management | 45 |
| Total | 365 |
In total, we estimate around 365 hours to deliver milestone-3, the workload will be spread amongst two developers and we will be able to deliver within 2 months.
Milestone 4 - Optimal Usage of Protocol Resources
Kubernetes extensions, operators and plugins, that optimise use of resources related to web3 protocols. Operators is the pattern to implement human knowledge and tasks into code - think of an experienced team who knows how to deploy and run the web3 protocol nodes / clients, watching for specific metrics or dashboards to track the status of the overall system and take actions if necessary.
| Deliverable | Hours Est. |
|---|---|
| Kubernetes Operators | 360 |
| Kubernetes Plugins | 120 |
| Project Management | 60 |
| Total | 540 |
In total, we estimate around 540 hours to deliver milestone-4, the workload will be spread amongst two developers and we will be able to deliver within 3 months.
Milestone 5 - Kubernetes Open Service Broker
Service brokers manage the lifecycle of services, and platforms interact with service brokers to provision, get access to and manage the services they offer. The Open Service Broker API defines these interactions. Such a capability in OpenKube allows usage of resources hosted by external providers / platforms, e.g. Filebase and Textile, via web3 protocols.
| Deliverable | Hours Est. |
|---|---|
| Integration with Pinning Services | 40 |
| Integration with Storage Providers | 120 |
| Project Management | 40 |
| Total | 200 |
In total, we estimate around 200 hours to deliver milestone-5, the workload will be spread amongst two developers and we will be able to deliver within 1.5 months.
Total Budget Requested
With an hourly rate of 75 USD for advanced developer with K8s, Java and Go experience (well below market rate) we estimate 2,409 total hours resulting in a total cost of 180,675 USD
| Deliverable | Hours Est. | Cost |
|---|---|---|
| milestone-1 | 224 | 16,800 USD |
| milestone-2 | 1080 | 81,000 USD |
| milestone-3 | 365 | 27,375 USD |
| milestone-4 | 540 | 40,500 USD |
| milestone-5 | 200 | 15,000 USD |
| Total | 2,409 | 180,675 USD |
Maintenance and Upgrade Plans
We plan to build more software libraries and applications on top of this spec in the future which will ensure the codebase will be in use and will receive periodic maintenance.
Team
Below we describe the core team behind this ****dev-grant proposal, we do know that there will be a few external contributors for peer review and architectural inputs. These individuals shall be acknowledged when components has been delivered.
Team Members
| Name | Background | |
|---|---|---|
| Bernardo | Computer Engineering background. Performed roles such as Software Developer, Technical Leader, Software Architect, Project Manager, Technical Product Manager in several areas: Mobile, Financial / Payment Services, Aerospace / Space, Document Management Systems, and Defence. Current architect at a public defence body working with design and implementation of data processing at very large scale. | https://www.linkedin.com/in/bernardo-wilde-sampaio-6583313/ |
| Renato | Computer Science background, previous telecom systems architect and senior UNIX systems administrator. Current senior development operations engineer focusing on complex information systems. | https://www.linkedin.com/in/rarruda/ |
| Paulo | Current Senior Full Stack Engineer at Doji, with extensive experience in various frontend frameworks and java backend development. Previous consultant in complex systems analysis. | https://www.linkedin.com/in/paulo-silva-filho-9584852a/ |
Team Member LinkedIn Profiles
Team Website
Relevant Experience
-
Bernardo:
- Design and implementation of embedded systems
- Design and implementation of mission critical / large-scale / high performance software systems
- Programming – C/C++ (20+ yrs), Java (20+ yrs), Python (6 yrs), Go (2 yrs)
- Security Programming – PKI, OpenSSL, Netscape Security Services (NSS), Bouncy Castle
- Enterprise / Solution / System architecture
- Project management using SCRUM, and PMP
-
Renato:
- Senior UNIX Systems Administrator
- Senior Development Operations Engineer
- Programming C/C++ (15+ yrs), Java (15+ yrs), Go (2 yrs)
-
Paulo:
- Advanced architecture experience (15+ yrs)
- Programming C/C++ (15+ yrs), Java (15+ yrs) and Matlab.
Team code repositories
PoC will be soon available in a Github repo.
Additional Information
We have potential users of this dev-grant using the systems that will be built on top of this operator - currently a handful of medium sized consultancy companies has signed up interest in using the underlying systems in order to access distributed resource protocol such as IPFS.
Further, these open source components may be used in a larger commercial application that is focusing on making Web3 Resource Protocols more widely accessible for individual developers and companies.
For further information and continuous updates on maintenance and wider open source components please visit https://www.openkube.org
