containers icon indicating copy to clipboard operation
containers copied to clipboard

[bitnami/spark] Cluster mode starting Apache Spark. Add valid value history

Open talosu opened this issue 3 years ago • 1 comments

Name and Version

bitnami/spark:*

What is the problem this feature will solve?

Easy to view logs and code optimization, etc.

Reference documentation: https://spark.apache.org/docs/latest/monitoring.html#viewing-after-the-fact

What is the feature you are proposing to solve the problem?

You can use this command: spark-class org.apache.spark.deploy.history.HistoryServer

What alternatives have you considered?

I can also customize the HistoryServer myself using apache/spark

talosu avatar Jul 22 '22 11:07 talosu

Hi @ererermao

Thanks a lot for your request but I am not sure if I understood it very well (and please correct me if I am wrong). Currently history-server.sh is available in our spark containers and you can start an spark container running history-server with the following command:

$ docker run -it --name spark --rm bitnami/spark:3.3 -- /opt/bitnami/scripts/spark/entrypoint.sh /opt/bitnami/spark/sbin/start-history-server.sh

To configure it properly please refer to Spark configuration.

fmulero avatar Jul 25 '22 10:07 fmulero

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] avatar Aug 13 '22 01:08 github-actions[bot]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

github-actions[bot] avatar Aug 19 '22 01:08 github-actions[bot]