datahub
datahub copied to clipboard
No data for Min and Max when enable profiling
Below is my Configuration for enabling profiling: profiling: enabled: True profile_pattern: allow: - "ods_mos_pvg.xxx"
After ingesting there are statistics for Null Count,Distinct Count but there is no data for Min | Max | Mean | Median
for a bug report to be actionable, could you provide more information, like source, datahub version? and verify that the column is created as a numeric type?
@xiphl Datahub version is 0.8.36. Source is hive table but it is in apache hudi format. There is no column has the statistics for Min | Max | Mean | Median even they are numeric type. is it because they are hudi . Below is my yml file:
source: type: sqlalchemy config: schema_pattern: allow: - test table_pattern: allow: - test.part profiling: enabled: True profile_pattern: allow: - "test.part" platform: presto connect_uri: presto://xxx:xxxx@xxxxx:8443 options: connect_args: host: xxxx port: 8443 catalog: hive schema: test protocol: https requests_kwargs: verify: False domain: "urn:li:domain:hive_profile": allow: - ".*"
sink: type: datahub-rest config: server: 'http://localhost:8080'
This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io
This issue was closed because it has been inactive for 30 days since being marked as stale.