miniflare icon indicating copy to clipboard operation
miniflare copied to clipboard

Package: Analytics Engine

Open CraigglesO opened this issue 3 years ago • 3 comments

Resolves https://github.com/cloudflare/workers-sdk/issues/4383

TODO:

  • [x] QUANTILEWEIGHTED function
  • [x] parse "INTERVAL 'X' DAY" and build a function
  • [x] vitest tests
  • [x] jest tests
  • [x] move sql exec into AnalyticsEngine constructor
  • [x] fetch support
  • [x] formatting

NOTES:

  • Miniflare api includes async getAnalyticsEngine(name: string): Promise<AnalyticsEngine>
  • shared-test-environment global state includes async queryMiniflareAnalyticsEngine(query: string): Promise<any> for jest and vitest.
  • http-server package function createRequestListener includes /cdn-cgi/mf/analytics_engine/sql
  • sqlite doesn't have strong aggregate support, so the implementation of QUANTILEWEIGHTED is pretty weak against large scale data.

CraigglesO avatar Oct 01 '22 05:10 CraigglesO

⚠️ No Changeset found

Latest commit: f93f54ec911342c85f1e8878dccf57f448b4171a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

changeset-bot[bot] avatar Oct 01 '22 05:10 changeset-bot[bot]

@CraigglesO Hey, you were asking about how quantileWeighted is implemented. Right now it uses https://clickhouse.com/docs/en/sql-reference/aggregate-functions/reference/quantileexactweighted

46bit avatar Oct 03 '22 12:10 46bit

Hi @CraigglesO, thanks for contributing this. We've been talking internally about whether SQLite is 'good enough' as a model for how Analytics Engine will behave. I think it's fine - the behaviour might not quite match production, but this will allow people to do testing locally and that's definitely a good thing. Happy to merge once some of the above issues are addressed :)

46bit avatar Oct 14 '22 14:10 46bit