feat: initial PR for adding security to edgex via openziti

[dovholuknf]

EdgexFoundry has elected to adopt OpenZiti for additional security. This is an initial attempt at what integrating github.com/openziti/ziti into ekuiper for EdgexFoundry might look like. (I tried my best to follow the commit guidelines, but I'm sure I didn't do something quite right, apologies in advance)

This PR adds functionality to ekuiper. It allows ekuiper to be removed from the underlay network if desired. In this PR the restPort is no longer available on the underlay (IP-based) network as it is exclusively accessible via OpenZiti if enabled.

Considerations/Notes:

• Logging - OpenZiti uses pfxlog which uses the logrus.StandardLogger() and had to be adapted to in order to allow OpenZiti logging in the ekuiper logs for debugging
• VaultSecret - edgex uses vault to deliver vault tokens to services. this vault token can be exchanged for a jwt which is then usable as authentication to the OpenZiti overlay network and other edgex services. A small VaultSecret was created to keep the vault token fresh and obtain new jwt's on a regular interval
• http clients have been adapted to allow connecting/sending data over the OpenZiti overlay network if enabled using the environment variables EDGEX_CREDENTIALS, EDGEX_CREDENTIAL_NAME, OPENZITI_CONTROLLER. If this is not desired, I'm happy to rework it how you see fit.
• http server ListenAndServe/ListenAndServeTLS are changed to Serve(listener)/ServeTLS(listener) respectively. http server is able to be taken off the IP-based underlay network entirely (making it unattackable by conventional IP-based attacks)

[ngjaying]

@dovholuknf Thanks. Please follow the Details link in DCO check to sign the commit.

[dovholuknf]

@dovholuknf Thanks. Please follow the Details link in DCO check to sign the commit.

Ahhh shoot. I forgot. Thanks

[dovholuknf]

I have fixed the DCO issue. Thanks

[dovholuknf]

sorry i've been away from this for a long long time... i'm nearly through the edgex work... i am hoping to return to this this week or next week.

[dovholuknf]

This should be ready to re-review now. If you would like to see instructions on testing with edgex - just let me know and i can provide a set of docker steps here

[dovholuknf]

~~the linter failed to run but it looks like it was some kind of error? can you maybe rerun it?~~ I found the errors appear to be red herrings. ran gofumpt...

also is the test case failure due to something i did? I'm surprised to see that based on what i added?

--- FAIL: TestExtensions (0.03s)
    mock_topo.go:392: Drop stream fails: ext is not found.
    mock_topo.go:392: Drop stream fails: ext2 is not found.
    plugin_rule_test.go:81: The test bucket size is 2.

i'll try to fix the 'max commit len' issue -- 58 chars at the moment? ;/

[ngjaying]

the linter failed to run but it looks like it was some kind of error? can you maybe rerun it?

also is the test case failure due to something i did? I'm surprised to see that based on what i added?

--- FAIL: TestExtensions (0.03s)
    mock_topo.go:392: Drop stream fails: ext is not found.
    mock_topo.go:392: Drop stream fails: ext2 is not found.
    plugin_rule_test.go:81: The test bucket size is 2.

i'll try to fix the 'max commit len' issue -- 58 chars at the moment? ;/

OK， we can try to fix lint and ut problems

[dovholuknf]

OK， we can try to fix lint and ut problems

much appreciated. i found the linter error by looking at the raw logs. the "UI view" just showed all the errors and confused me. i ran gofumpt and hopefully appeased the linter.

[dovholuknf]

i think i figured out how to run gci and gofumpt and hopefully the linter will be happy this time

[dovholuknf]

plugin was built with a different version of package github.com/lf-edge/ekuiper/internal/conf/logger (previous failure)

I do think I somehow broke that test now that i see this. I changed this file. I'm definitely going to ask for help figuring out how to fix that. Also thanks for pushing the 'approve' button on all the linter commits. I sure hope I have them all sorted now :(

[ngjaying]

plugin was built with a different version of package github.com/lf-edge/ekuiper/internal/conf/logger (previous failure)

I do think I somehow broke that test now that i see this. I changed this file. I'm definitely going to ask for help figuring out how to fix that. Also thanks for pushing the 'approve' button on all the linter commits. I sure hope I have them all sorted now :(

Sure, I'll take a look later today but not now since I am busy with other things in hand. Once you have a commit merged, the checks will be run without a need of approval.

[ngjaying]

Build packages error

The default build has edgex tag, the build package check will run the built kuiperd to check. Then the caErr cause a panic in openziti.go AuthenicatedContext. I guess there are some dependency services which are not existed in the test env. Is EdgeX still have secure and non-secure mode? Maybe we need to have a configuration in eKuiper to switch between secure and non-secure and default to non-secure to pass the tests. Thus, I make a commit for that to add enableOpenZiti config.

UT error

It is caused by the logger adaptor. It seems to me that the adaptor is unneccessary to apply to the logger globally. We can decorate it only when calling open ziti functions. Thus, I make a commit for that.

Now all checks are passed except the coverage is too low. Is it possible to add ut? (Looks hard to me because of external dependencies).

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 5.11811% with 241 lines in your changes are missing coverage. Please review.

Project coverage is 63.17%. Comparing base (d5bbd74) to head (ef120d3). Report is 12 commits behind head on master.

Files	Patch %	Lines
internal/edgex/vault.go	0.00%	138 Missing :warning:
internal/edgex/logger_adaptor.go	0.00%	35 Missing :warning:
internal/edgex/openziti.go	0.00%	31 Missing :warning:
internal/io/http/client_edgex.go	23.08%	19 Missing and 1 partial :warning:
internal/server/server_edgex.go	18.18%	9 Missing :warning:
internal/server/server.go	0.00%	8 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2603      +/-   ##
==========================================
- Coverage   63.49%   63.17%   -0.32%     
==========================================
  Files         339      345       +6     
  Lines       38932    39283     +351     
==========================================
+ Hits        24717    24816      +99     
- Misses      12046    12295     +249     
- Partials     2169     2172       +3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[ngjaying]

@dovholuknf Thank you. All looks good now. Is it possible to add more unit tests? If not, I think we can merge for now. And we'll need doc updates in another PR.

[dovholuknf]

@dovholuknf Thank you. All looks good now. Is it possible to add more unit tests? If not, I think we can merge for now. And we'll need doc updates in another PR.

Thank you for the help! I'd really like to say yes to adding tests but I'm really strapped trying to get this and all the other edgex work out the door. i will put it on my backlog though and maybe in the coming weeks I can add them? Unit tests aren't probably too useful here other than maybe making sure configuration is set and defaults processed since it really needs an openziti overlay -- if that makes sense? I'm happy to look at it in the future, when I can get to it. Right now it's tight. :(

[dovholuknf]

I'll need help with what kind of doc and 'how/where' to doc. If you have any pointers on that, please let me know! Maybe make an issue and assign it to me? -- same for test now that i think about it?

[dovholuknf]

and one related question - will a docker container be produced from this repo that I could reference in the edgex project? sorry for all the comments...

[ngjaying]

and one related question - will a docker container be produced from this repo that I could reference in the edgex project? sorry for all the comments...

Yes, we can trigger an alpha release, that would produce a docker image.

[ngjaying]

I'll need help with what kind of doc and 'how/where' to doc. If you have any pointers on that, please let me know! Maybe make an issue and assign it to me? -- same for test now that i think about it?

OK, I'll create an issue for that.

[dovholuknf]

Yes, we can trigger an alpha release

that would be much appreciated. I looked but it doesn't seem like one exists yet. Should I use GH Discussions for this request? Add a new issue? or just keep commenting here? :) thanks again

[ngjaying]

Yes, we can trigger an alpha release

that would be much appreciated. I looked but it doesn't seem like one exists yet. Should I use GH Discussions for this request? Add a new issue? or just keep commenting here? :) thanks again

@dovholuknf https://github.com/lf-edge/ekuiper/releases/tag/v1.14.0-alpha.2 tagged. Discussions thread is OK for me.