Wild-Storage
Wild-Storage copied to clipboard
Use YouTube as Infinite Cloud Storage
Wild-Storage
Wild-Storage lets you use YouTube as a Infinite Cloud Storage. It embeds any files(Images, Videos, Documents, etc) into a video. The video can be uploaded to YouTube and the file can be retrieved from the video. A expansion & compression technique is used to defeat YouTube's compression algorithm. The file can be retrieved from the video without any loss of data.
Demo Video (YouTube Link)
~~Virtual Environment Set Up~~ ❌ (... Not Required Now)
Create Virtual Environment
python3 -m venv env
Activate Virtual Environment
source env/bin/activate
Install Required Packages
pip install -r requirements.txt
~~Compile C++ Library & Generate Python Bindings~~ ❌ (... Not Required Now)
g++ -fPIC -shared DataProccessing/cLibs/helper.cpp -o DataProccessing/cLibs/c_lib.so
Wild-Storage CLI App ✅
Make the script executable
chmod +x cli_app
Run CLI App
Make sure GCC
compiler, Python 3
, pip
, Python venv
is installed.
Now open terminal in the project directory and run the following command to complete the necessary setup and run the app.
./cli_app
How to Use? ✅
Wait for the app to start.
Create video from files
- Copy all your files into
\data\input
directory - Select
Encode file to video
option on the app - Wait for the process to complete
-
Generated video (
op.mp4
) & Metadata (metadata.json
) will be available in\data\generated
directory - Upload the video to YouTube and copy-paste the contents of
metadata.json
in description of the video - Make sure the uploaded video is
public
/unlisted
Retrieve files from video
- Select
Decode file from video
option on the app - Enter the youtube video url (Make sure the uploaded video is
public
/unlisted
) - Wait for the process to complete
- The files will be available in
\data\retrieved
directory
Cleaning up
- Select
Clean
option on the app to clean data directory
Reload Time
- Select
Reload time
option to set the time to wait before reloading the app
Exit
- Select
Exit
option on the app to exit
Use Dockerized CLI App ✅ ✅
You just need to have Docker
installed on your system.
Build Docker Image
docker build -t wild-storage .
Run Docker Container
docker run -it wild-storage
Run Dockerized CLI App
./cli_app
Copy Input Files from Host to Container
You can copy files from host to container using the following command. You can copy folders and files.
docker cp <host_path> <container_id>:app/data/input
Copy Output Files from Container to Host
The video and metadata files can be copied from container to host using the following command.
docker cp <container_id>:app/data/generated <host_path>
Copy Retrieved Files from Container to Host
The retrieved files can be copied from container to host using the following command.
docker cp <container_id>:app/data/retrieved <host_path>
DataProcessing Working
Embed File Into Video
Retrieve File From Video
Expansion And Compression
Python Wrapped C++ Library
Wild-Storage was a complete Python Implementation. Beign a very slow interpreted language it was taking too much time to process files. Later I moved some heavy task codebase to C++ and wrapped those in Python using ctypes. This helped to process the files ~3x faster.
Features
- Parallel Processing using multiple CPU cores is supported now.
- C++ Library is used for heavy tasks to make the process faster.
Credits
This project is inspired by Infinite-Storage-Glitch which is a Rust implementation. I liked the concept and tried to implement in my favourite language Python. The working of both may not be same.