EMR Serverless supported?

[set92]

Describe the bug I have done the installation on a jupyter notebook with an EMR Serverless 7.7.0. But when I try to access the URL from the Spark UI, it shows a white screen with only This request is not valid. Please generate a new URL..

Environment spark version: 3.5 platform: EMR

To Reproduce Steps to reproduce the behavior:

1. Create EMR Serverless application.
2. Open a Notebook, and run %%configure -f { "conf": { "spark.jars.packages" : "io.dataflint:spark_2.12:0.2.9", "spark.plugins": "io.dataflint.spark.SparkDataflintPlugin", "spark.sql.maxMetadataStringLength": "1000", "spark.dataflint.iceberg.autoCatalogDiscovery": "true", } }
3. After running spark on another cell, you will get the link to access the Spark UI (https://spark-live-ui.emr-serverless.amazonaws.com/), which in my case takes me to https://j-00fqdv663u738h0a.dashboard.emr-serverless.us-east-1.amazonaws.com/.
4. Try to access https://j-00fqdv663u738h0a.dashboard.emr-serverless.us-east-1.amazonaws.com/dataflint will return the error mentioned.

Expected behavior Be able to access the interface.