datahub icon indicating copy to clipboard operation
datahub copied to clipboard

feat(graph) Add createdOn, createdActor, updatedOn, updatedActor to graph edges

Open chriscollins3456 opened this issue 2 years ago • 2 comments

This PR adds the mechanism to define what the createdOn and updatedOn timestamps are on a graph index edge as well as the createdActor and updatedActor. In order to specify, all you need to do is set the path to the field on the aspect inside of the Relationship annotation. As an example, this is what you can do to set these fields on Upstream in UpstreamLineage

  /**
   * The upstream dataset the lineage points to
   */
  @Relationship = {
    "name": "DownstreamOf",
    "entityTypes": [ "dataset" ],
    "isLineage": true,
    "createdOn": "upstreams/*/created/time"
    "createdActor": "upstreams/*/created/actor"
    "updatedOn": "upstreams/*/auditStamp/time"
    "updatedActor": "upstreams/*/auditStamp/actor"
  }
  @Searchable = {
    "fieldName": "upstreams",
    "fieldType": "URN",
    "queryByDefault": false
  }
  dataset: DatasetUrn

This also sets a default createdOn and createdActor on all new graph edges being created using the metadataChangeLog.systemMetadata.lastObserved timestamp and the metadataChangeLog.created.actor for those fields.

Checklist

  • [ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • [ ] Links to related issues (if applicable)
  • [ ] Tests for the changes have been added/updated (if applicable)
  • [ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • [ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

chriscollins3456 avatar Dec 02 '22 17:12 chriscollins3456

Unit Test Results (build & test)

622 tests  +1   618 :heavy_check_mark: +1   16m 13s :stopwatch: -9s 158 suites +1       4 :zzz: ±0  158 files   +1       0 :x: ±0 

Results for commit 1e8c13a8. ± Comparison against base commit 4f7b5ac5.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Dec 02 '22 18:12 github-actions[bot]

Aside from build failing :)

jjoyce0510 avatar Dec 03 '22 01:12 jjoyce0510