quickstarts icon indicating copy to clipboard operation
quickstarts copied to clipboard

Published 20 hours ago • verified publisher icon airbytehq

Added the quickstart of performance Optimization stack

Open utsavdotdev opened this issue 1 year ago • 5 comments

Solved this issue #29

What I have done?

  • Airbyte Setup: Established Datadog as the data source and BigQuery as the destination for raw metrics and logs extraction.
  • dbt Transformation: Introduced dbt models (daily_cpu_usage, daily_error_logs, hourly_user_activity) for structured analysis.
  • Dagster Orchestration: Set up a Dagster pipeline, ensuring seamless data flow from Airbyte extraction to dbt transformations.

utsavdotdev avatar Oct 18 '23 13:10 utsavdotdev

Sorry @ThaliaBarrera, I have made a blunder mistake. There is limitation for datadog. As I am in hurry I haven't test it.

utsavdotdev avatar Oct 21 '23 04:10 utsavdotdev

Sorry @ThaliaBarrera, I have made a blunder mistake. There is limitation for datadog. As I am in hurry I haven't test it.

No worries @utsavbhattarai007! All your code seems correct. Would you like to use the UI instead of Terraform and try to complete it, if you have a Datadog account with source data? Otherwise, you can take another quickstart. You already did the biggest effort of understanding what to do, etc. Let me know!

ThaliaBarrera avatar Oct 21 '23 09:10 ThaliaBarrera

Yes, I would definitely do it but As I am out of town, I am unable to do it.

utsavdotdev avatar Oct 21 '23 09:10 utsavdotdev

Okay @utsavbhattarai007! Maybe when you're back, if the hackathon is still running you can complete it. I'll be happy to review it then.

ThaliaBarrera avatar Oct 23 '23 14:10 ThaliaBarrera

Yes, I will definitely complete it.

utsavdotdev avatar Oct 23 '23 15:10 utsavdotdev