Performance Bottleneck with 200,000 Policy Records

[Harishs27]

Title: Performance Bottleneck with 200,000 Policy Records

Description: We are experiencing significant performance issues when using Casbin with a large number of policy records (approximately 200,000). Here are the details of the problem:

Casbin Version: 0.4.1

Spring Boot Version: 2.6.6 Environment: Cloud based microservice applications, using alpine os and using java 11 Adapter: JDBC

Steps to Reproduce:

Load 200,000 policy records into the database. Initialize Casbin enforcer with the policies. Perform authorization checks using the enforcer.

Observed Behavior:

The initial loading of policy records takes a considerable amount of time. Authorization checks are slow, impacting the performance of the application. Enforcer is taking almost 30 sec to authorize.

Expected Behavior:

Casbin should handle large datasets efficiently without significant performance degradation. Authorization checks should be quick and not noticeably impacted by the number of policy records.

Additional Information:

And since the policy are loaded in memory, the application is consuming more than 1.4GB out 2GB allocated memory. So if the application consumes more than 1.6GB application response will slow down and eventually app will crash

I am using RBAC with domains/tenants model

Thank you for your attention to this issue. Please let me know if any additional information is needed.

[casbin-bot]

@tangyang9464 @imp2002

[hsluoyz]

@Harishs27 see: https://casbin.org/docs/performance

[Harishs27]

@hsluoyz Just as explained in cloud or multi tenant applications. We have followed all the steps except sharding on enforcer.

Would it possible for the enforcer to reload the policies if any new tenant is onboarded.
If so, can you please provide me an example of how to achieve the same.
Where the policies are loaded among multiple enforcer and while validating permissions which enforcer to use and how do i reload policy if new tenant is onboarded.