spline-spark-agent
spline-spark-agent copied to clipboard
Removing unnecessary parameters from JDBC URIs
When spark is reading/writing with JDBC datasource spline reports its URI to lineage - that's greate because database and table are visible there
Unfortunately some non-datasource-identifing parameters are also added in that URI, for example:
jdbc:mysql://host:3336/my_database?serverTimezone=Poland&useSSL=false&characterEncoding=utf8:my_table
Not sure if there's generic way to remove such serverTimezone
, useSSL
, characterEncoding
parameters ? They may cause lineage problems if other job uses same datasource but with different URI parameters