Please support Spark progressbar monitor like in Zeppelin notebooks

[chanansh]

trafficstars

Problem

Cannot see long pyspark command progress,

Proposed Solution

see for comparison Zeppelin notebook and the package https://github.com/swan-cern/sparkmonitor (which does not work for me and repo is deprecated)

Additional context

When I run a pyspark action I would like to see a progress bar to make sure the command is alive and the cluster has not halted.