Sentinel
Sentinel copied to clipboard
alibaba
Add rule persistence support and refactor dashboard for better scalability
Describe what this PR does / why we need it
Sentinel dashboard provide web UI where we can setting rules, viewing machines, viewing monitoring, etc, it's very useful and convenient for user, especially new coming user.
By default, rules are stored in memory, and users may need to persist the rule by different own way. But at present, dashboard is a single module, and user-defined modification can only be made on the source code of the whole project, which is not very convenient for continuous upgrade and keep up with the latest version.
Does this pull request fix one issue?
Fixes #1241, #1257 Related #1375, #1415
Describe how you did it
To make it simple to use, a configuration parameter and a properties file is used to support one type of repository.For example, using nacos,
Add VM paramteter -Drule.repository.type=nacos,
Add a properties file Nacos.properties in the same folder path as sentinel-dashboard.jar, and in the properties file we can configure settings for Nacos.
Considering independence and extensibility, I tried to layer the dashboard project into the following separate modules:
- sentinel-dashboard-common
- sentinel-dashboard-entity
- sentinel-dashboard-repository
- sentinel-dashboard-repository-memory
- sentinel-dashboard-repository-redis
- sentinel-dashboard-repository-nacos
- sentinel-dashboard-repository-zookeeper
- sentinel-dashboard-repository-apollo
- sentinel-dashboard-service
- sentinel-dashboard-web
In sentinel-dashboard-common, common use class(config, util) , such as DashboardConfig,VersionUtils etc.
In sentinel-dashboard-entity , all entities used by dashboard, include rule entity, machine/app entity, metric entity.
In sentinel-dashboard-repository, based on DynamicRuleProvider/DynamicRulePublisher interface, abstract the common logic for persist rules with two abstract class AbstractRuleProvider/AbstractRulePublisher, methods are reserved for the realization by specific sub modules of different repository type.
-
AtractRuleProvider#fetchRules -
AbstractRulePublisher#publishRules
The encoder/decoder package include serialization/deserialization for rules of different type, and the logic is for common use.
The RuleKeyBuilder interface define method String buildRuleKey(String app, String ip, Integer port); for generate the persistent key of rule.
For example, the flow rule, in Nacos:
If it's applied to one machine instance, the rule key is appName-ip-port-flow-rules, such as
sentinel-dashboard-192.168.1.100-8719-flow-rules.
If we set the rule which take application as dimension, that is all machine instances of this application apply the same rule, if have four instances, we'll persist four properties with four key in Nacos, the keys are:
app-ip1-port1-flow-rules
app-ip2-port2-flow-rules
app-ip3-port3-flow-rules
app-ip4-port4-flow-rules
Pay attention to that the RuleKeyBuilder#buildRuleKey method is a default implementation to generate ruleKey, which is alternative, and user can customize it in specific repository modules.
In specific repository modules, such as sentinel-dashboard-repository-nacos,sentinel-dashboard-repository-redis etc, the implementation is simple, we just need to focus on how to get and push rules with the specific repository. For example, NacosRuleProvider/NacosRulePublisher class extends AbstractRuleProvider/AbstractRulePublisher class, and override fetchRules/publishRules method.
NacosProperties is used for reading properties from file.
@PropertySource(value = {"classpath:repository/nacos.properties", "file:nacos.properties"}, ignoreResourceNotFound = true)
Pay attention to that there are two files, the first file are in /resources/repository/nacos.properties, which is convenient for debug in IDE environment, and the second file nacos.properties is useful when package the sentinel-dashboard.jar, and it has higher priority.
In sentinel-dashboard-service, define vo(req/resp), api and implementation of CRUD functions for rule of different type, the key logic is encapsulated in AbstractRuleService class.
In sentinel-dashboard-web, are web UI(page/javascript/css), http RESTFul controller, web config and filters, it will be better if we split it into complete front and back-end separation in the furture.
Previous we have flow_v1,flow_v2, which may confuse new users, hope to support both set rules via machine or whole application, so the flow_v1,flow_v2 are merged into one file, in this way it's flexible and could meet most scenes.
Describe how to verify it
-
Package sentinel-dashboard.jar, test with different type of repository(Nacos,Redis,Zookeeper etc) For Nacos repository: 1.Start Nacos server 2.Start dashboard with
-Drule.repository.type=nacosCopy
nacos.propertiesfile fromsentinel-dashboard/src/main/resource/repository/nacos.propertiesto the same folder path of `sentinel-dashboard.jarnacos.properties:nacos.serverAddr=localhost nacos.sentinelGroup=SENTINEL_GROUPNote that if Nacos server is in the same machine as sentinel-dashboard.jar, and use default setting , the
nacos.propertiescan be omitted.Start dashboard
java -jar -Dproject.name=sentinel-dashboard -Dcsp.sentinel.dashboard.server=localhost:8080
-Drule.repository.type=nacos sentinel-dashboard.jar3.Test with dashboard application itself, test CRUD functions for rules of different type in dashboard app, and view the rules in Nacos web console.
4.Test with demo project, start
sentinel-demo-annotation-spring-aopapplication with two different instances, test CRUD functions for rules of different type via application or single machine. -
Test in IDE environment
-
Run test cases
Special notes for reviews
Now persistence for five types of rules have been completed(flow/degrade/system/authority/paramFlow), persistence for the gateway rule and cluster have not implement yet.
With the design of sentinel-dashboard-repository and sentinel-dashboard-service, I find it easy to implement specific repository except sentinel-dashboard-repository-memory, since it needs to communicate with sentinel-transport http api. For simplicity, I add id/app/ip/port fields in AbstractRule , but this class is in core module. Is there any better way, please give some help.
A new module sentinel-transport-client is added, in order to implement sentinel-dashboard-repository-memory, I prefer put the client module in transport module. In this way, it is an independent module, and easy to use and expand.
Pay attention to the id field of RuleEntity, previous version use AtomicLong ids and increment one when adding a new rule. This may have some problems when dashboard restart and memory data lost. To solve this I add IdGenerator interface and DefaultIdGenerator class, to generate unique and self increasing id for a new rule.
Codecov Report
Merging #1453 into master will not change coverage. The diff coverage is
n/a.
@@ Coverage Diff @@
## master #1453 +/- ##
=========================================
Coverage 44.13% 44.13%
Complexity 1726 1726
=========================================
Files 376 376
Lines 10708 10708
Branches 1417 1417
=========================================
Hits 4726 4726
Misses 5415 5415
Partials 567 567
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 8594729...8594729. Read the comment docs.
Could you please resolve conflicts?
Conflicts has been resolved.
A demo for sentinel-datasource-redis module is added, which is used for demonstrate and test
using RedisDataSource and persistent rules in redis via dashboard, PTAL.
We can test with following steps:
- Start local redis server
- Start RedisDataSourceDemo1
- Start RedisDataSourceDemo2
- Start dashboard:
-Dproject.name=sentinel-dashboard -Dcsp.sentinel.dashboard.server=localhost:8080 -Drule.repository.type=redis - Watch the console of demo1 and demo2, add and modify rules in dashboard, view rules persisted in redis, restart demo or dashboard, test and watch.
Note: In step2, step3, needn't add VM argument since the dashboard server address and some other parameters have been configured via code in demo.
More test cases and document need to be added, I'll try to improve them later.
Codecov Report
Merging #1453 into master will increase coverage by
0.39%. The diff coverage is87.03%.
@@ Coverage Diff @@
## master #1453 +/- ##
============================================
+ Coverage 43.96% 44.36% +0.39%
- Complexity 1719 1750 +31
============================================
Files 376 376
Lines 10660 10752 +92
Branches 1418 1426 +8
============================================
+ Hits 4687 4770 +83
- Misses 5408 5414 +6
- Partials 565 568 +3
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 46076b3...b1668df. Read the comment docs.
请问 网关的现在实现了吗？是否是我的gateway服务加入了 -Dcsp.sentinel.app.type=1 ，成为一个网关的服务，那么他的操作是不具有持久化的
你好。我在使用过的过程中，你描述的 五大规则的确可以。非常棒。但是现在我想同时在网关层（gateway）做流控，会抛出异常：CommandNotFoundException: /gateway/getApiDefinitions 。增加了五大规则的持久化，不应该影响网关使用异常
加入了 -Dcsp.sentinel.app.type=1 ，成为一个网关的服务 抛出异常：CommandNotFoundException: /gateway/getApiDefinitions
记得没有修改网关规则处理，理论上不影响使用，只是规则是存在内存里的。
这个异常提示是缺少相关command的处理包，看看应用端是否有加sentinel-gateway-adapter相关的依赖，比如使用SpringCloudGateway的sentinel-spring-cloud-gateway-adapter, Zuul1的sentinel-zuul-adapter, Zuul2的sentinel-zuul2-adapter。网关command的处理是在它们的父依赖包sentinel-api-gateway-adapter-common下。
网关规则的持久化暂时还没实现，因为跟其它规则有些不同，有API分组和网关流控规则，还不太确定之前的抽象能否满足，还有集群流控规则也还没实现，社区看看一起讨论设计下哈，如果项目中有使用问题请多反馈和讨论。
@sczyh30 Sorry for late. Merged from master and the conflicts have been resolved, please check.
Codecov Report
Merging #1453 into master will decrease coverage by
0.22%. The diff coverage is87.03%.
@@ Coverage Diff @@
## master #1453 +/- ##
============================================
- Coverage 44.58% 44.36% -0.23%
+ Complexity 1804 1750 -54
============================================
Files 382 376 -6
Lines 10912 10752 -160
Branches 1467 1426 -41
============================================
- Hits 4865 4770 -95
+ Misses 5451 5414 -37
+ Partials 596 568 -28
| Impacted Files | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| ...a/csp/sentinel/slots/system/SystemRuleManager.java | 70.39% <83.33%> (+5.39%) |
21.00 <2.00> (+2.00) |
|
| ...alibaba/csp/sentinel/slots/block/AbstractRule.java | 70.73% <100.00%> (+12.11%) |
22.00 <8.00> (+8.00) |
|
| ...ource/spring/cloud/config/SentinelRuleLocator.java | 36.71% <0.00%> (-0.98%) |
5.00% <0.00%> (ø%) |
|
| ...nterceptor/AbstractSentinelInterceptorSupport.java | |||
| ...n/cdi/interceptor/SentinelResourceInterceptor.java | |||
| ...ployment/SentinelJaxRsQuarkusAdapterProcessor.java | |||
| ...tion/cdi/interceptor/ResourceMetadataRegistry.java | |||
| ...inel/annotation/cdi/interceptor/MethodWrapper.java | |||
| ...ent/SentinelAnnotationQuarkusAdapterProcessor.java | |||
| ...a/com/alibaba/csp/sentinel/node/StatisticNode.java | 51.11% <0.00%> (+1.11%) |
26.00% <0.00%> (+1.00%) |
|
| ... and 2 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update b91305a...058eb26. Read the comment docs.
ParamFlowRuleEntity 这个类中代码 public ParamFlowRuleEntity(ParamFlowRule rule) { AssertUtil.notNull(rule, "Authority rule should not be null"); this.rule = rule; } 是不是不小心写错单词了
ParamFlowRuleEntity(ParamFlowRule rule)
确实写错了，该段代码在PR已注释掉没有使用，上面的ParamFlowRuleEntity fromAuthorityRule(String app, String ip, Integer port, ParamFlowRule rule)也是拼写错误。
@huyoufu 感谢指出问题，找时间我去把这里修正完善
Master code has been merged and conflicts fixed, PTAL. @sczyh30 @jasonjoo2010 @linlinisme