docker_datalake
docker_datalake copied to clipboard
vincentnam
Datalake
How to make an automatic implementation of workflow from standard definition Need for : - security - code testing - authentication and restriction (ex : deny data access on disks...
# Datalake enhancement : modular backend and optimization Block storage: overlay Native object storage Block mode overlay ? Is it possible to change the storage backend (Openstack Swift) in the...
Add reverse proxy before every access to service in the architecture.
What will be done : - [ ] Code cleaning - [ ] Apache Airflow dag - [ ] Python scripts - [ ] Comments - [ ] Apache Airflow...
# Data management : Data management is composed of 2 parts : batch data and stream data. The stream part is not currently well implemented in the architecture. Tools are...