Open Grant Proposal: SXX Cloud

[sx-star]

Open Grant Proposal: SXX Cloud - WEB3 Infrastructure Storage Architecture tools

Name of Project SXX Cloud

Proposal Category: devtools-libraries

Proposer: sx-star

(Optional) Technical Sponsor: @Lawrence

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT, APACHE2, or GPL licenses?: "Yes"

Project Description

SXX Cloud builds tools and infrastructure for developers to easily create and manage content on IPFS, back up permanently to Filecoin, which simplifies decentralized storage, brings your content to the world and stores it forever. The product mainly has the following functions:

• Realize the management capability of basic Metadata Service. The main capability of Metadata Service is to provide a standard S3 interface externally, so that enterprise developers or individuals can access it at the minimum cost, and can connect old users who use various cloud services. The system switches with one key.
• Implement basic Storage Service. Manage the underlying data, such as creating metadata copies, hash verification, transparent transmission and other functions. The main technology stack includes Kafka message queue, md5 data verification, etc.
• Implement Auth Service.The service mainly stores fixed Access Key and Secret Key, and uses SK to verify whether the API request is legal
• Realize functions such as file status change, hot and cold conversion, encryption and decryption. Hot storage refers to IPFS, cold storage refers to Filecoin, IPFS is temporary storage, and Filecoin is long-term backup. This service will first smash the files, build MerkelDAG, and store them in IPFS; then it will trade with the Storage Provider for cold storage; because IPFS files are time-sensitive, when IPFS files expire, they can be recovered from cold storage.
• Implement traffic entry management and billing system. This service will perform statistics and billing on requests based on the frequency, number of times, and the amount of data requested.
• Build the Chain-Sync Service. The service uses the subscription mode to filter the messages of the trusted chain in real time, and track the system dependent data in real time.
• Tenant Service. Implement functions such as centralized management of blank window quotas, AK/SK, and Api quotas.

SXX Cloud aims to be scalable, high-performance, and highly available. On the basis of often fully and vertically splitting the functional roles of each layer, the core service capabilities are horizontally expanded in the form of micro-services to achieve commercial-level object storage capabilities. Main technology stack: Clickhouse, Cassandra, Kafka, Ceph cluster, MySQL, Redis, Keycloak, Vault, etc. Among them, Clickhouse is used to store a large amount of log data and cold data, Cassandra is used to solve the KeyValue performance problem of object storage, and Kafka is used to realize the communication at all levels of the system and balance the system load in the time dimension.

Value

SXX Cloud provides support by layering and is divided into multiple microservices for construction. It not only contains related components of Web2, but also many services of Web3. Through the mutual invocation of components, the system has a greater degree of flexibility, so that it can respond quickly to changes in business requirements, and the system has good scalability. This application is mainly for the Web3 part, including functions such as Metadata Management, IPFS and Filecoin Storage, and Data Transfer.

Deliverables

SXX Cloud builds tools and foundational architectures for enterprises and developers to easily create and manage content on IPFS, backed up to Filecoin permanently.

SXX Cloud delivers the following:

• Tenant management functions: Provide registration channels for enterprises and developers, enterprises and users can implement AK/SK management, API quota management and other functions
• Provides APIs that conform to the S3 standard protocol. Enterprises or developers can use the AWS SDK to access the SXX Cloud API to implement bucket management, authentication, file access, IPFS, and Filecon hot and cold interchange.
• Provide metadata management functions. Enables users to manage the underlying data, such as metadata copying, hash checking, access control, and more.

S3 Protocol API:

SXX Cloud architecture is designed to be independent of programming language, using S3-supported interfaces to store and retrieve objects.

With the REST API, you can create, extract, and delete buckets and objects using standard HTTP requests. To use the REST API, you can use any HTTP-enabled toolkit. As long as the objects are anonymously readable, you can even use a browser to extract them.

The REST API uses standard HTTP headers and status codes to make standard browsers and toolkits work as expected. In some areas, we've added functionality to HTTP, and in these cases we'll do our best to match the new features we've added to the standard HTTP usage style.

Development Roadmap

Milestone1 - Project Boundary and Architecture

Goal:

• Determine the boundaries of the project
• Architecture design, technology stack selection
• Determine acceptance criteria
• Define components and scale
• Define and design workflows

Expected Hours:

• 50

Expected Days:

• 15

Expected Budget:

• $5,000 USD

Milestone2 - Achieving Basic Metadata

Goal:

• Developers can access the system through the S3 protocol
• Implement the s3 protocol standard interface
• Bucket, metadata management
• Meta、Header、Put、Delete、Copy Implement

Expected Hours:

• 100

Expected Days:

• 30

Expected Budget:

• $10,000 USD

Milestone3 - Basic Storage Capabilities

Goal:

• Implement object storage to ipfs, and access, encrypt, decrypt, pin and other operations on objects through the S3 protocol
• Hash check, Head transparent transmission, data cache management
• Data block management
• IPFS Data block management
• Provide data encryption and decryption capabilities

Expected Hours:

• 130

Expected Days:

• 40

Expected Budget:

• $13,000 USD

Milestone4 - Implement file status changes

Goal:

• Realize the mutual transfer between ipfs and filecoin storage IPFS DAG Calc
• Trading with Filecoin SP
• Retrieve files based on dealId
• Interchange between ipfs and filecoin files, providing a file state change interface

Expected Hours:

• 80

Expected Days:

• 20

Expected Budget:

• $8,000 USD

Total Budget Requested

Milestone	Work Time	Duration	Budget
Milestone 1 - Project Boundary and Architecture	50 hours	15 days	$5,000
Milestone 2 - Achieving Basic Metadata	100 hours	30 days	$10,000
Milestone 3 - Basic Storage Capabilities	130 hours	40 days	$13,000
Milestone 4 - Implement file status changes	80 hours	20 days	$8,000
Total			$36,000

Maintenance and Upgrade Plans

This project is designed to provide an IPFS/Filecoin-based development framework for enterprises and developers. We will continue to update the project code iteratively to ensure the sustainable evolution and development of the project. We welcome the participation of like-minded developers from all over the world to provide us with constructive suggestions, which we will carefully refer to and adopt.

Team

Team Members

• Leo
• GeekTR
• Zhijie Huang
• Steven

Team Member LinkedIn Profiles

• Leo : https://github.com/leotons
• GeekTR : https://github.com/geektheripper/
• Zhijie Huang : https://github.com/jhyehuang
• Steven : https://github.com/hukaibaihu

Team Website

Relevant Experience

• Leo, Back-end R&D engineer, involved in blockchain related R&D for many years, used to be a core R&D member of virtual currency exchange.
• GeekTR, senior system architect, proficient in local and cloud-based infrastructure. Years of experience in designing integrated business systems.
• Zhijie Huang, proficient in blockchain technology and common encryption algorithms, and build enterprise-level blockchain applications based on ETH, quroum and Fabric for a long time.
• Steven, 8 years of front-end development experience, senior web developer. Master React, Vue and other development frameworks, familiar with common data structures and algorithms, master React Native, Cordova and other cross-end App development technologies, have front-end architecture capabilities, and promote architecture optimization.

Team code repositories

Cold Storage tools, you can simply view it as the previous version of SXX Cloud

• https://git.sxxfuture.net/publique/cold-storage

Additional Information

• Get information about Open Grant from https://grants.filecoin.io/ , and Slack channel #grants-help
• If you have any questions, please contact me. Mail : [email protected] / Slack : @SXX Future-Xindy
• More information about us , see https://www.sxxfuture.com/