datahub icon indicating copy to clipboard operation
datahub copied to clipboard

feat(ingest): add new ingestion source PowerBI Report Server

Open alcoccoque opened this issue 2 years ago • 2 comments

This PR adds a new ingestion source PowerBI Report Server.

Metadata that can be ingested:

  • report name
  • report description
  • data sources
  • ownership(add an existing user with graphql queries or creates new)
  • transfer folders structure to DataHub as it is in Report Server
  • webUrl to report in Report Server

Due to the limits of PBIRS REST API, it's impossible to ingest following data for now:

  • tiles info
  • full datasource of report
  • dataset of the report

Next types of reports can be ingested:

  • PowerBI report(.pbix)
  • Paginated report(.rdl)
  • Mobile report
  • Linked report

Mapping: Report -> Dashboard

Why PowerBI Report Server is a new ingest module?

  • PowerBI Report Server has its API to retrieve metadata( PowerBI Cloud uses another API)
  • PowerBI Report Server has different authentication compared to PowerBI Cloud
  • PowerBI Report Server and PowerBI Cloud are just different sources, Report Server can sync reports with Cloud but it works only manually and for specific reports, so you can't substitute Report Server with Cloud

Checklist

  • [ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • [ ] Links to related issues (if applicable)
  • [ ] Tests for the changes have been added/updated (if applicable)
  • [ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • [ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

alcoccoque avatar Jul 11 '22 12:07 alcoccoque

Unit Test Results (build & test)

599 tests  ±0   595 :heavy_check_mark: ±0   12m 1s :stopwatch: -2s 149 suites ±0       4 :zzz: ±0  149 files   ±0       0 :x: ±0 

Results for commit e91a0585. ± Comparison against base commit 0dc2d6af.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 11 '22 17:07 github-actions[bot]

Unit Test Results (metadata ingestion)

       8 files  ±0         8 suites  ±0   56m 36s :stopwatch: +58s    757 tests +1     754 :heavy_check_mark: +3  3 :zzz: ±0  0 :x:  - 2  1 516 runs  +2  1 510 :heavy_check_mark: +4  6 :zzz: ±0  0 :x:  - 2 

Results for commit e91a0585. ± Comparison against base commit 0dc2d6af.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 11 '22 17:07 github-actions[bot]

Hi @alcoccoque Could you please look at the issue reported on the datahub slack https://datahubspace.slack.com/archives/CUMUWQU66/p1676377452280029

siddiquebagwan avatar Feb 14 '23 12:02 siddiquebagwan