Spring Boot Cloud Config

[lony2003]

Description

A new library that implement a backend of Spring Cloud Config.

Originally from https://github.com/fangkehou-team/dapr-spring, this library created a backend of SpringCloudConfig just like SpringCloudVault. The original library only uses secret store as config store api is not stable at that time. As the configuration api is stable now, the config loader using that api has been implemented, but subscription of configuration has not implemented, just for reservation.

Issue reference

this PR will close: #1225

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

• [x] Code compiles correctly
• [x] Created/updated tests
• [x] Extended the documentation

[salaboy]

@lony2003 first of all, thanks so so much for contributing back with this. This is highly appreciated.. I will start adding comments in the PR with small details that we need to fix and questions about how this would work.

It will be great if you can add as part of this PR an example on the spring-boot-examples/ directory showing how this functionality would work. We probably also need tests inside sdk-tests to demonstrate that this works from a client point of view. You will notice that we are heavily relying on Testcontainers for all of this, so it will be quite cool to see if the testcontainers integration is missing something to support this functionality.

[lony2003]

@lony2003 first of all, thanks so so much for contributing back with this. This is highly appreciated.. I will start adding comments in the PR with small details that we need to fix and questions about how this would work.

It will be great if you can add as part of this PR an example on the spring-boot-examples/ directory showing how this functionality would work. We probably also need tests inside sdk-tests to demonstrate that this works from a client point of view. You will notice that we are heavily relying on Testcontainers for all of this, so it will be quite cool to see if the testcontainers integration is missing something to support this functionality.

Thanks. the code is not completed currently, I created this pull request is for the problems in issue dapr#1225 , I think we need a schema for cloud config url, and have some other things to be discussed😊😊😊

[lony2003]

Cloud Config Import Schemas

Secret Store Component

url structure

dapr:secret:<store-name>[/<secret-name>][?<paramters>]

paramters

parameter	description	default	available
type	value type	value	value/doc
doc-type	type of doc	properties	yaml/properties/or any file extensions you want

• when type = value, if secret-name is specified, will treat secret as the value of property, and secret-name as the key of property; if none secret-name is specified, will get bulk secret and treat every value of secret as the value of property, and every key of secret as the key of property.
• when type = doc, if secret-name is specified, will treat secret as a bunch of property, and load it with property or yaml loader; if none secret-name is specified, will get bulk secret and and treat every value of secret as bunches of property, and load them with property or yaml loader.
• secret store with multiValud = true must specify nestedSeparator = ".", and using type = doc is not recommanded

demo

multiValued = false:

store content(file secret store as example)

{
	"dapr.spring.demo-config-secret.singlevalue": "testvalue",
	"multivalue-properties": "dapr.spring.demo-config-secret.multivalue.v1=spring\ndapr.spring.demo-config-secret.multivalue.v2=dapr",
	"multivalue-yaml": "dapr:\n  spring:\n    demo-config-secret:\n      multivalue:\n        v3: cloud"
}

valid demo url

• dapr:secret:democonfig/multivalue-properties?type=doc&doc-type=properties
• dapr:secret:democonfig/multivalue-yaml?type=doc&doc-type=yaml
• dapr:secret:democonfig/dapr.spring.demo-config.singlevalue?type=value
• dapr:secret:democonfig?type=value
• dapr:secret:democonfig?type=doc

multiValued = true, nestedSeparator = ".":

store content(file secret store as example)

{
	"value1": {
		"dapr": {
			"spring": {
				"demo-config-secret": {
					"multivalue": {
						"v4": "config"
					}
				}
			}
		}
	}
}

will be read as

{
	"value1": {
		"dapr.spring.demo-config-secret.multivalue.v4": "config"
	}
}

valid demo url

• dapr:secret:demo-config-multi/value1?type=value
• dapr:secret:demo-config-multi?type=value

Configuration Component

url structure

dapr:config:<store-name>[/<key>][?<paramters>]

paramters

parameter	description	default	available
type	value type	value	doc/value
doc-type	type of doc	properties	yaml/properties/or any file extensions you want
subscribe	subscribe this configuration	false	true/false

• when subscribe = true, will subscribe update for the configuration.
• when type = value, if key is specified, will treat config value as the value of property, and key as the key of property; if none key is specified, will get all key and value in the config-name and treat every config value as the value of property, and every key as the key of property.
• when type = doc, if key is specified, will treat config value as a bunch of property, and load it with property or yaml loader; if none key is specified, will get all key and value in the config-name and treat every config value as bunches of property, and load them with property or yaml loader.

Demo

store content(table as example)

key	value
dapr.spring.demo-config-config.singlevalue	testvalue
multivalue-properties	dapr.spring.demo-config-config.multivalue.v1=spring\ndapr.spring.demo-config-config.multivalue.v2=dapr
multivalue-yaml	dapr:\n spring:\n demo-config-config:\n multivalue:\n v3: cloud

valid demo url

• dapr:config:democonfigconf/dapr.spring.demo-config-config.singlevalue?type=value
• dapr:config:democonfigconf/multivalue-properties?type=doc&doc-type=properties
• dapr:config:democonfigconf/multivalue-yaml?type=doc&doc-type=yaml
• dapr:config:democonfigconf?type=doc
• dapr:config:democonfigconf?type=value

[lony2003]

All jobs have done, waiting for review. Note that subscription of configuration api is not implemented, just as a reservation.

[lony2003]

I have checked the error, and it seems that jacoco coverage throws the error. I have add a test of bulk secret to coverage more situation in DaprSecretStoreConfigDataLoader.java listed in the first line of codeconv report. Not sure if it's the cause of jacoco coverage failed, but I can't add more coverage as it's a little difficult to test error situation in bootstrap stage (springboot application will crash and test will be failed)

[salaboy]

@lony2003 this is amazing! thank so much for all the work that you put on this PR! I really appreciate the demo application in the spring boot examples, but I have one more ask. Can you make sure that this also works on Kubernetes? I want to make sure that people can understand how this will work on Kubernetes when compared with the local version.. Following something like this: https://github.com/dapr/java-sdk/tree/master/spring-boot-examples/kubernetes what are the steps that people will need to perform on a cluster to configure Spring Cloud Config inside Kubernetes, assuming that the config parameters are in Kubernetes ConfigMaps or Secrets. Does this make sense?

[lony2003]

@salaboy thank you for review.

Can you make sure that this also works on Kubernetes?

Yes, as long as you configure the secret store components and configuration components in your dapr runtime. Dapr Cloud Config just a client that query the secret store and configuration using java sdk.

I want to make sure that people can understand how this will work on Kubernetes when compared with the local version.

There is no difference of what does this library do in Kubernates and local version. Just make a rpc request to the dapr runtime, and waiting to get the result.

what are the steps that people will need to perform on a cluster to configure Spring Cloud Config inside Kubernetes, assuming that the config parameters are in Kubernetes ConfigMaps or Secrets.

if you want to use cloud config on clusters, you define a dapr component that represent a secretstore or configuration component that using Kubernates builtin feature, and have the store name configured in the springboot configuration using the schema above, and store the secret or configuration in SpringBoot style (yaml, properties, etc. like the demo above), and it works (if java-sdk didn't perform differently between Kubernates and local version of course).

I'm not that familiar with Kubernates, but if dapr runtime has the same return type compared to Kubernates and local versions, it will work at the same way.

[salaboy]

@lony2003 I totally see that, but because Dapr usually runs on Kubernetes, it will be great to add the steps needed in the kubernetes directory so the same example works there. Notice how the producer and consumer apps have the instructions to create the container needed and we have the manifest to run on Kubernetes.. can you copy that for the config demo? I think running the Kubernetes tutorial will give you enough context to just copy what you need for the config demo.

[salaboy]

@lony2003 if you don't feel comfortable with that.. I can spend some time after KubeCon (the whole next week) to see if I can get it working on your branch

[lony2003]

@salaboy I'll try to enhence that part. But may be I need your help as I really don't familiar with that😂😂😂.

I am from China, so sometimes I might use some inappropriate words when communicating in English. I hope you don't mind😃😃😃.

[salaboy]

@lony2003 your english is more than fine! thanks so much for all the work, if you have questions don't hesitate to ask. Are you on discord right?

[lony2003]

Are you on discord right?

@salaboy yeah, I'll communicate with you on discord if I meet troubles. Thanks!

[lony2003]

@salaboy I have checked the dapr documentation and find that Kubernetes ConfigMap is not supported in dapr runtime, only Kubernetes Secret supported in dapr secret.

I have write a simple documentation of how to deploy on Kubernetes. Further test and review may needed.

[lony2003]

@salaboy I have tried the Kubernetes usage and everything needed is documented, it works for me.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 73.65591% with 98 lines in your changes missing coverage. Please review.

Project coverage is 75.31%. Comparing base (d759c53) to head (70d4eea). Report is 138 commits behind head on master.

Files with missing lines	Patch %	Lines
...igdata/secret/DaprSecretStoreConfigDataLoader.java	68.25%	15 Missing and 5 partials :warning:
...g/DaprConfigurationConfigDataLocationResolver.java	70.68%	10 Missing and 7 partials :warning:
...data/config/DaprConfigurationConfigDataLoader.java	68.08%	10 Missing and 5 partials :warning:
...oudconfig/config/DaprCloudConfigClientManager.java	48.00%	7 Missing and 6 partials :warning:
...ret/DaprSecretStoreConfigDataLocationResolver.java	85.71%	5 Missing and 3 partials :warning:
...onfig/configdata/DaprCloudConfigParserHandler.java	85.00%	3 Missing and 3 partials :warning:
...ta/config/DaprConfigurationConfigDataResource.java	60.00%	6 Missing :warning:
...data/secret/DaprSecretStoreConfigDataResource.java	57.14%	6 Missing :warning:
.../cloudconfig/config/DaprCloudConfigProperties.java	76.47%	4 Missing :warning:
...t/examples/cloudconfig/CloudConfigApplication.java	33.33%	2 Missing :warning:
... and 1 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #1230      +/-   ##
============================================
- Coverage     76.91%   75.31%   -1.60%     
- Complexity     1592     1707     +115     
============================================
  Files           145      211      +66     
  Lines          4843     5473     +630     
  Branches        562      594      +32     
============================================
+ Hits           3725     4122     +397     
- Misses          821     1004     +183     
- Partials        297      347      +50     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[salaboy]

@dapr/maintainers-java-sdk I believe that we can merge this PR, we will incrementally improve the quality of this PR, but I want this module to be present in the SDK. Please approve, we can fix any findings in smaller PRs.

[artur-ciocanu]

@lony2003 also could you please sync your branch withmaster branch, your branch is out of date.

[lony2003]

@artur-ciocanu I am working on my graduation currently, once I have time, I‘ll resolve them.

@salaboy Thank you for approving this.

[salaboy]

@lony2003 as soon as you have the changes done please let me know

[salaboy]

@artur-ciocanu thank so much for all the reviews and unblocking

[cicoyle]

@lony2003 - can you take a look at CI?

[lony2003]

@cicoyle I'm working on this, I just merged the base and haven't changed my code. I'll notify you once I have finished

[salaboy]

@lony2003 can you take this back, I can work with you to get this reviewed and merged by the maintainers