CERN Database and Analytics Group

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

cerndb

Hadoop Profiler, or hprofiler, is a tool which is able to analyze on- and off-CPU workloads on distributed computing environments.

cerndb

Tool for gathering blocks and replicas meta data from HDFS. It also builds a heat map showing how replicas are distributed along disks and nodes.

cerndb

Spark-Dashboard is a solution for monitoring Apache Spark jobs. This repository provides the tooling and configuration for deploying an Apache Spark Performance Dashboard using containers technology.

cerndb

Code and links to the data for the article "Machine Learning Pipelines with Modern Big DataTools for High Energy Physics"

cerndb

Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems...

cerndb