Scaling Access Control Lists for multi-million users

[arosenberg01]

I recently found Casbin while searching for help with managing Access Control List-type permissions for a multi-million user base where the policies need to be updatable in real-time. It looks like a great fit for the business logic I want to implement, but the suggested persistence patterns around loading filtered policies / Watchers is a little unintuitive.

For a standard client-facing API server, my initial thought process would be something like: initialize an enforcer with a db adapter (Mongo or Postgres) on app startup - when a user makes a request for a resource, set a Filter for policies related to that user/resource and call enforcer.LoadFilteredPolicy(filter) before doing the actual enforcer.enforce() check(s). That way, whenever my database gets updated with a policy change (by a separate enforcer), the next enforcer.LoadFilteredPolicy() call will pick it up, no?

Are there some big performance implications I'm missing? I'm a bit confused by the the need for all of the Watcher-based synchronization if I can just fetch the relevant policy lines that I care about from the database for each API request. Nearly every workload I can imagine for Casbin would be skewed super read-heavy, so generous indexing seems appropriate and performant.

Highlighting any gaps in my understanding would be much appreciated! The project looks great overall.

[hsluoyz]

@nodece @GopherJ

[nodece]

@arosenberg01 Your understanding is correct.

For read-only policy use enforcer.LoadFilteredPolicy(), I think it is great.

Maybe enforcer unable to update policy correctly when use filterAdapter, something like:

• insert duplicate policy

I'm a bit confused by the the need for all of the Watcher-based synchronization if I can just fetch the relevant policy lines that I care about from the database for each API request.

We supports disable the Watcher.

[0x8f701]

@arosenberg01

Are there some big performance implications I'm missing? I'm a bit confused by the the need for all of the Watcher-based synchronization if I can just fetch the relevant policy lines that I care about from the database for each API request. Nearly every workload I can imagine for Casbin would be skewed super read-heavy, so generous indexing seems appropriate and performant.

I also would like to implement a dynamic filter, but I think it's not that easy to find relevant policies, I don't have a detailed idea on it.

[hsluoyz]

Is this still active?

[nodece]

yes. For ACL, We might need to consider incremental loading policy.

[HridayK97]

@hsluoyz

Piggybacking on this since it is semi-related. Trying to get a better understanding of how casbin performs when using ACL. I'm running in a serverless environment with AWS Lambda, so I can't have the casbin enforcer loaded and always running constantly. So I have to retrieve a filtered policy set from my database (Will do this with SQL) and then initialize and enforce casbin each request.

Let's say I have a simple model that just matches r.sub == p.sub && r.obj == p.obj && r.act == p.act

If I have a list of policies, what is the difference between me iterating through the policy list and performing that comparison check myself vs loading the policies into casbin and then calling the enforce function?

From my brief testing, it seems that manually iterating through my policies is much faster. For 15k policies, I'm seeing a difference of 80ms vs 20ms. Am I missing something?

Appreciate the help!

[hsluoyz]

@HridayK97 how did you do iterating through the policy list and performing that comparison check myself ? Can you give some code?