🔬 A data science oriented container launcher 🔬

Our instance of Onyxia @ INSEE - Onyxia Web - Onyxia UI - Onyxia K8s API

Onyxia is a web app that aims at being the glue between multiple open source backend technologies to provide a state of art working environnement for data scientists.
Onyxia is developed by the French National institute of statistics and economic studies (INSEE).

Core feature set:

• An interface for launching docker images (e.g: Jupyter, RStudio) on demand on a Kubernetes cluster.
  The catalog of available images is not part of the app, you can create your own. (here is the catalog we build for the institute's needs.)
• Users can define the amount of RAM, CPU and GPU they would like to allocate to their containers.
• Specify a custom init script to be executed at launch.
• Define environnement variables to be made available in the containers.
• Save and restore your service service configurations
• Deep integration with S3 for working with data (S3 as the open standard, not the AWS service) and with Vault (for secret management)
• Keycloak integration.

Launching a container

https://user-images.githubusercontent.com/6702424/152631131-44050af8-a979-4c25-b09a-1a5521558361.mp4

S3 File explorer

https://user-images.githubusercontent.com/6702424/161458858-57321269-5a10-42e3-971a-80a505928fb5.mp4

Vault secret manager

For injecting secret environment variables in the containers.

https://user-images.githubusercontent.com/6702424/154877930-ce5dab0b-e508-43b5-a3d5-51bd6105ac45.mov

Table of content

• Table of content
  • Screenshots
  • Deploy onyxia on your infrastructure today 🚀
  • Installation & configuration
  • Modules
    • Services catalogs
    • Docker images for services
    • Cloudshell
    • Miscellaneous
    • Infrastructure scripts
  • Media
  • Roadmap 🛣
    • Recently released 🎁
    • Coming soon ☄️
    • WIP 🏗
    • Ideas 💡

Screenshots

Deploy onyxia on your infrastructure today 🚀

The simplest way to install Onyxia is to use Helm.

helm repo add inseefrlab https://inseefrlab.github.io/helm-charts
helm install onyxia inseefrlab/onyxia --set ingress.enabled=true --set ingress.hosts[0].host=datalab.yourdomain.com

Browse to http://datalab.yourdomain.com and enjoy :)

Note that this is only a bare installation of Onyxia with some major limitations (no authentication, deployed services won't be accessible ...). Read below for more configuration options.

Installation & configuration

See Installation

Modules

Onyxia is split into several modules :

Services catalogs

Onyxia relies on catalogs to provide users with a selection of services they can install in one click.
You can either create your own repositories or use the default ones :

Docker images for services

InseeFRLab maintains various Docker images that extends standard images so that they work nicely inside the datalab.
You can browse them here : Repositories using docker-image tag on InseeFRLab

Cloudshell

Onyxia integrates a cloudshell that is based on a WebSSH docker image.
The docker image used is codenamed Shelly and is available here : Shelly

Miscellaneous

Infrastructure scripts

The cloud-scripts repository is a collection of scripts we used at some point at Insee. They are provided as is with minimal to no documentation and support. They are, currently at least, used as memo and not production grade code.
The repository is available here : cloud-scripts

Media

Energy Data Hack

OpenLAB - Download slides

Article d'acteurs publics

Les Entrepreuneurs d'intérêt général - Découvrir le programme

Roadmap 🛣

The Onyxia project is actively developed. We are constantly working on new functionalities to meet our users needs at Insee. Do not hesitate to get in touch with us to ask questions or share your ideas!

Recently released 🎁

• New services: Argo CD, Argo Workflow, Pinot
• Step by step Onyxia deployment guide
• Customizable UI themes
• Onyxia installation documentation
• Project documentation (CONTRIBUTING.md...)

Coming soon ☄️

• Projects and collaboration
• Transform File Explorer into Data Explorer

WIP 🏗

• Onyxia installation documentation
• Project documentation (CONTRIBUTING.md...)
• New UI for FileExplorer

Ideas 💡

• End user documentation

• Extend the catalog of data science services

• Data governance (data & metadata management, data cataloging, data lineage, data quality management)

• What data management features does a user need in Onyxia (objects explorer, PV manager...)?

• Billing, monitoring & housekeeping of services

• Onyxia deployment automation

• Instance administration (users & groups...)