posthog

posthog copied to clipboard

Problem

Selecting from events is slow, if filtering by a person filter on the event.

https://github.com/PostHog/posthog/issues/22427

We can build HogQL level solutions (or tools?) to make this easy.

Changes

• Creates a system to inline filters into lazy table subqueries.
• I'm effectively recycling Robbie's SessionMinTimestampWhereClauseExtractor
• Still some cleanup todo regarding magic constants
• All of this will be in a new HogQL modifier that's disabled by default.

Does this work well for both Cloud and self-hosted?

Yes

How did you test this code?

With a query like:

SELECT
    event
FROM
    events
WHERE
    and(ilike(person.properties.email, '%[email protected]%'), less(timestamp, toDateTime('2024-05-23 09:47:35.499176')), greater(timestamp, toDateTime('2024-05-22 09:47:30.500146')))
ORDER BY
    timestamp DESC
LIMIT 101
OFFSET 0

I also added one test to see that the ilike clause gets duplicated.

Size Change: 0 B

Total Size: 1.06 MB

ℹ️ View Unchanged

Filename	Size
frontend/dist/toolbar.js	1.06 MB

_{compressed-size-action}

📸 UI snapshots have been updated

12 snapshot changes in total. 0 added, 12 modified, 0 deleted:

• chromium: 0 added, 12 modified, 0 deleted (wasn't pushed!)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

I think I addressed the questions/comments here, so ready for a re-review.

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

• chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

• ‼️ NotImplementedError: FieldAliasType.resolve_database_field not implemented /api/projects/{parent_lookup_team_id}/insights/... View Issue
• ‼️ NotImplementedError: FieldAliasType.resolve_database_field not implemented /api/projects/{parent_lookup_team_id}/insights/... View Issue
• ‼️ NotImplementedError: FieldAliasType.resolve_database_field not implemented /api/projects/{parent_lookup_team_id}/insights/... View Issue
• ‼️ NotImplementedError: FieldAliasType.resolve_database_field not implemented /api/projects/{parent_lookup_team_id}/insights/... View Issue
• ‼️ NotImplementedError: IsTimeOrIntervalConstantVisitor has no method visit_tuple posthog.tasks.tasks.process_query_task View Issue

_{Did you find this useful? React with a 👍 or 👎}