sparkmonitor
sparkmonitor copied to clipboard
swan-cern
An extension for Jupyter Lab & Jupyter Notebook to monitor Apache Spark (pyspark) from notebooks
I am trying to use sparkmonitor for monitoring my spark job which i am running on jupyterlab and spark on kubernetes kernel.It works fine when i am using python3 kernel....
when click `Restart the kernel, then re-run the whole notebook` ,just throw this error. ``` Exception in thread Thread-3: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner...
it work fine with spark session simple demo  but if I test the hive query in jupyter lab with sparkmonitor extention enabled. the ```python from pyspark.sql import SparkSession app_name="pyspark-hive-demo"...
the old monitor's feature  the new:  So where is the feature which is in old ver but not in the new one.
Hi, Noticed the following issue when restarting the kernel from a classic Jupyter Notebook using JEG to launch remote spark kernels in kubernetes. The sparkmonitor doesn't show up, and in...
## Summary of Issue _Note - Related container log error:_ `ERROR JupyterSparkMonitorListener: Exception sending socket message: ` Hello! We are running a Jupyterhub service that spawns Kubernetes pods for single...
Tested with Azure HDInsight Cluster 4.1.4.8 / PySpark 2.4.6
Hi there! I installed the sparkmonitor extension into a dockerimage that is based on jupyterhub: ``` FROM jupyterhub/k8s-singleuser-sample:1.2.0 RUN pip install pyspark==3.2.0 RUN pip install delta-spark==1.1.0 USER root RUN apt...
In the task timeline tab of the Spark monitor, when a task fails and the user clicks on the task bar to see information about the task, it is not...
Hi, Noticed the following issue when restarting the kernel from a classic Jupyter Notebook using JEG to launch remote spark kernels in kubernetes. The sparkmonitor doesn't show up, and in...