Access to the StreamingQuery status through the ingress

[nickpodobiedov]

We use SparkOperator and have an issue with access to the detailed stream statistics.

1. Operator configuration: '-ingress-url-format=http://<ingress_url>/spark/{{$appName}}'
2. Sparkapplication configuration:

    spark.ui.proxyBase: /spark/pyspark-realtime
    spark.ui.proxyRedirectUri: /spark/pyspark-realtime
    spark.ui.reverseProxyUrl: /spark/pyspark-realtime

I can get access to the all application data from UI, for example: http://<ingress_url>/spark/pyspark-realtime/jobs/ e.t.c. Also I can get Structured Streaming page: Its URL is https://<ingress_url>/spark/pyspark-realtime/StreamingQuery/. But when I want to see details for each stream, I got redirect to the https://<ingress_url>/StreamingQuery/statistics/?id=<stream_id> without needed prefix. I think spark.ui.proxyRedirectUri param is responsive for this, but I cannot rewrite it, even with my configuration (when I setup this parameter directly) I get the spark.ui.proxyRedirectUri=/ in driver configmap. Thank you for help!