feat: add project secret api keys
Problem
Customers have explicitly asked for project-level API keys, so when their employees leave the company their API keys are not breaking their apps.
But we also already have a Feature Flags use case that has a token that is not tied to a user. (long term, we would move that over to project secret API keys)
Closes #843
Changes
Introducing a ProjectSecretAPIKey model, along with a new Permission that only allows specific scopes (we don't want this key type to allow all endpoints, read the discussion in the RFC)
Also implements this auth/permission on the Endpoints model, so that we see it in action. Will happily break this out into a separate PR after validating approach here.
Pretty much clones the frontend approach we have for PersonalAPIKeys (we can refactor PersonalAPIKeys to reuse this later on).
How did you test this code?
- lots of unit tests
π Stay up-to-date with PostHog coding conventions for a smoother review.
Changelog: (features only) Is this feature complete?
Okay, useful input, thanks gents @haacked & @benjackwhite π©
What if we go about it this way - only allow this key type (SDK naming TBC) on specific actions that make our customers' apps run, i.e. endpoint execution, FF local evaluation, and such.
Configuration (CRUD) is done by humans, execution by machines.
This way, we exclusively specify the permission_ and authentication_ classes on the route that needs it, we avoid creating a new ViewSet for these endpoints, we don't have to worry about the created_by_X issue, and we don't add the condition inside the TeamAndOrgViewSetMixin?
The model LGTM and covers the feedback I shared in https://github.com/PostHog/product-internal/pull/843. Will leave the more important permissioning points to the folks with more context.
What if we go about it this way - only allow this key type (SDK naming TBC) on specific actions that make our customers' apps run, i.e. endpoint execution, FF local evaluation, and such.
What's "endpoint execution" in this context?
Size Change: +697 B (+0.02%)
Total Size: 3.7 MB
βΉοΈ View Unchanged
| Filename | Size | Change |
|---|---|---|
frontend/dist/toolbar.js |
3.7 MB | +697 B (+0.02%) |
What's "endpoint execution" in this context?
@haacked, I'm referring to the new Endpoints product that we're building. Essentially a pre-defined query meant to power B2B2C dashboards.
Endpoint execution is the /api/projects/2/endpoints/event-count/run route. See the changed files in products/endpoints/backend of this PR for more.
πΈ UI snapshots have been updated
78 snapshot changes in total. 0 added, 78 modified, 0 deleted:
chromium: 0 added, 78 modified, 0 deleted (diff for shard 8, diff for shard 14, diff for shard 1, diff for shard 11)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
- #39865
π (View in Graphite) master
This stack of pull requests is managed by Graphite. Learn more about stacking.
How strongly do customers want this? I can totally see the benefit, but I think there is also a major drawback to introducing new keys for users to understand, which I've already seen happening with the feature flags key we introduced.
We could make the UX of creating personal api keys for a team much better, or even create them by default for a team. We could also handle transferring the team scoped keys in the UX for removing a team member from a team to avoid the user leaves and everything breaks problem.
Having tokens team scoped feels like the right thing to do, but not sure if it's worth the added complexity for users and for us if there are other ways around it to keep request.user working nicely.
πΈ UI snapshots have been updated
24 snapshot changes in total. 0 added, 24 modified, 0 deleted:
chromium: 0 added, 24 modified, 0 deleted (diff for shard 14, diff for shard 8, diff for shard 1)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
πΈ UI snapshots have been updated
1 snapshot changes in total. 0 added, 1 modified, 0 deleted:
chromium: 0 added, 1 modified, 0 deleted (diff for shard 11)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
πΈ UI snapshots have been updated
1 snapshot changes in total. 0 added, 1 modified, 0 deleted:
chromium: 0 added, 1 modified, 0 deleted (diff for shard 11)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
@benjackwhite thanks for a detailed review π Should be in a much better place now.
A bit of a self-reflection - I should've been much more explicit about the type of input I was looking from you (and others) here. I don't want you to waste time reviewing code that's not cleaned up for review yet - don't think it's the best use of your time. I'll be clearer when I'm looking for directional input (ie "yeah, this approach works better" or "nope, still doesn't sit right with me"), before I spend time cleaning things up. :))
What do we think of this now?
The new approach is something I actually really like, its just an extension of the feature flag key but more flexible. I'm onboard π
FYI there's a refactor in #40246 that means this PR will need an additional check in auth.py. More context at https://github.com/PostHog/posthog/pull/40246#discussion_r2457135990
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label β otherwise this will be closed in another week. If you want to permanentely keep it open, use the waiting label.
πΈ UI snapshots have been updated
3 snapshot changes in total. 0 added, 3 modified, 0 deleted:
chromium: 0 added, 3 modified, 0 deleted (diff for shard 14, diff for shard 1)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
πΈ UI snapshots have been updated
2 snapshot changes in total. 0 added, 2 modified, 0 deleted:
chromium: 0 added, 2 modified, 0 deleted (diff for shard 14)webkit: 0 added, 0 modified, 0 deleted
Triggered by this commit.
Migration SQL Changes
Hey π, we've detected some migrations on this PR. Here's the SQL output for each migration, make sure they make sense:
posthog/migrations/0939_projectsecretapikey.py
BEGIN;
--
-- Create model ProjectSecretAPIKey
--
CREATE TABLE "posthog_projectsecretapikey" ("id" varchar(50) NOT NULL PRIMARY KEY, "label" varchar(40) NOT NULL, "mask_value" varchar(11) NULL, "secure_value" varchar(300) NULL UNIQUE, "created_at" timestamp with time zone NOT NULL, "last_used_at" timestamp with time zone NULL, "last_rolled_at" timestamp with time zone NULL, "scopes" varchar(100)[] NULL, "created_by_id" integer NULL, "team_id" integer NOT NULL);
--
-- Create constraint unique_team_label on model projectsecretapikey
--
ALTER TABLE "posthog_projectsecretapikey" ADD CONSTRAINT "unique_team_label" UNIQUE ("team_id", "label");
ALTER TABLE "posthog_projectsecretapikey" ADD CONSTRAINT "posthog_projectsecre_created_by_id_4aa6c6a3_fk_posthog_u" FOREIGN KEY ("created_by_id") REFERENCES "posthog_user" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "posthog_projectsecretapikey" ADD CONSTRAINT "posthog_projectsecretapikey_team_id_57c2fc1a_fk_posthog_team_id" FOREIGN KEY ("team_id") REFERENCES "posthog_team" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_projectsecretapikey_id_c862daf3_like" ON "posthog_projectsecretapikey" ("id" varchar_pattern_ops);
CREATE INDEX "posthog_projectsecretapikey_secure_value_084560df_like" ON "posthog_projectsecretapikey" ("secure_value" varchar_pattern_ops);
CREATE INDEX "posthog_projectsecretapikey_created_by_id_4aa6c6a3" ON "posthog_projectsecretapikey" ("created_by_id");
CREATE INDEX "posthog_projectsecretapikey_team_id_57c2fc1a" ON "posthog_projectsecretapikey" ("team_id");
CREATE INDEX "posthog_pro_team_id_d4c925_idx" ON "posthog_projectsecretapikey" ("team_id", "created_at");
COMMIT;
Last updated: 2025-12-11 12:45 UTC (62b50bc)
π Migration Risk Analysis
We've analyzed your migrations for potential risks.
Summary: 1 Safe | 0 Needs Review | 0 Blocked
β Safe
Brief or no lock, backwards compatible
posthog.0939_projectsecretapikey
ββ #1 β
CreateModel
Creating new table is safe
model: ProjectSecretAPIKey
β
βββ> βΉοΈ INFO:
βΉοΈ Skipped operations on newly created tables (empty tables
don't cause lock contention).
Last updated: 2025-12-11 12:46 UTC (62b50bc)
FYI there's a refactor in #40246 that means this PR will need an additional check in auth.py. More context at #40246 (comment)
When I update the filter to also let phs_ through, and I hit an endpoint that shouldn't be accessible with a Project Secret API Key, I get Authentication credentials were not provided.
Leaving that check as is, I get Personal API key found in request Authorization header is invalid. which makes more sense.
@joshsny any reason you can see that we must add the filter for phs_ next to pha_ in auth.py?
@haacked Are we cool to go with this as the first step without breaking feature flags in any way, especially considering your Rust implementation.
FYI there's a refactor in #40246 that means this PR will need an additional check in auth.py. More context at #40246 (comment)
When I update the filter to also let
phs_through, and I hit an endpoint that shouldn't be accessible with a Project Secret API Key, I getAuthentication credentials were not provided.Leaving that check as is, I get
Personal API key found in request Authorization header is invalid.which makes more sense.@joshsny any reason you can see that we must add the filter for
phs_next topha_in auth.py?
If I pass phs_ to the endpoint I'd expect to get something like "Project secret tokens are not supported for this endpoint" since we know it's not a personal api key, that's the only reason to add it here that I can see
@haacked Are we cool to go with this as the first step without breaking feature flags in any way, especially considering your Rust implementation.
Let me take a look. The good news is this won't affect /flags because flags doesn't look at personal api keys or secret tokens.
The rust implementation of /flags/definitions will need to be updated, but that's not yet released so I can do that later.
I assume you've tested this with /local_evaluation?
Just to confirm: this will eventually replace the existing Feature Flags Secure API Key? i.e. we'll migrate those keys to this new key type. Since it uses the same key prefix.
That would be my hope. It could happen in a follow-up
Wiz Scan Summary
| Scanner | Findings |
|---|---|
| - | |
| 1 |
|
| - | |
| - | |
| - | |
| Total | 1 |
To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.
Visual regression: Storybook UI snapshots updated
Mode: UPDATE (triggered by human commit 3a9ecea)
Changes: 2 snapshots (2 modified, 0 added, 0 deleted)
What this means:
- Snapshots have been automatically updated to match current rendering
- Next CI run will switch to CHECK mode to verify stability
- If snapshots change again, CHECK mode will fail (indicates flapping)
Next steps:
- Review the changes to ensure they're intentional
- Approve if changes match your expectations
- If unexpected, investigate component rendering
@haacked Thanks a bunchhhhh for that review! I think I've addressed all the issues now:
- activity logging
- filter on oauth, so we now get 'this endpoint does not support a project secret api key' type of error
- atomic last_used_at following approach on personal api key
- few more touch-ups on permissions
- double checked local_evaluation runs (locally)
@haacked do you know if these changes around ProjectSecretAPIKeyUser would shake up Django and impact FF's when we deploy?
do you know if these changes around ProjectSecretAPIKeyUser would shake up Django and impact FF's when we deploy?
I don't think it would. I can try and test this branch out soon, but we have a lot going on right now so probably won't get to it immediately.
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label β otherwise this will be closed in another week. If you want to permanentely keep it open, use the waiting label.
Visual regression: Storybook UI snapshots updated
Changes: 2 snapshots (2 modified, 0 added, 0 deleted)
What this means:
- Snapshots have been automatically updated to match current rendering
- Next CI run will switch to CHECK mode to verify stability
- If snapshots change again, CHECK mode will fail (indicates flapping)
Next steps:
- Review the changes to ensure they're intentional
- Approve if changes match your expectations
- If unexpected, investigate component rendering