Repo extensions

[anbraten]

closes #783

Allow users to set urls for extensions. Those extensions are little external webservice which can implement a set of endpoints to replace woodpecker internal functionality like:

• secrets storage
• registry storage
• pipeline config processing

Changes

• [x] wait for #916
• [x] add config, secrets & registry endpoint urls to repo settings
• [x] change extensions urls from repo settings
• [x] add docs

[Savemech]

heya, is there would be any updates?

[natto1784]

bump

[anbraten]

@qwerty287 Started updating this PR. For the first step I would just do some refactoring to load the different "services". And in a second step I would add the http service implementations for secrets & registries.

Currently we use some terms for "addons" like services, addons, plugins, extensions (what I've introduced when initially working on this). Do you have any preferences how should I call them? As the builtin implementations will also be of that ... type, sticking with services could be a quite fitting term?

What should I do with the go-plugin addon system for secrets, registries & configs?

[woodpecker-bot]

Deployment of preview was successful: https://woodpecker-ci-woodpecker-pr-915.surge.sh

[qwerty287]

Currently we use some terms for "addons" like services, addons, plugins, extensions (what I've introduced when initially working on this). Do you have any preferences how should I call them? As the builtin implementations will also be of that ... type, sticking with services could be a quite fitting term?

I'd say:

• Services: pipeline steps running the full time
• Addons: Functionality added to the server binary using hashicorp plugins (probably only forges)
• Plugins: pipeline images with special entrypoint
• Extensions: Everything else, so config, secrets, registry... extension

What should I do with the go-plugin addon system for secrets, registries & configs?

Just remove it would be fine for me, as it's refactored anyways.

[anbraten]

Okay, then I will rename the folder and the config section to extensions, right?

[anbraten]

Renamed everything as discussed. Only question now left is how the can unify the config fetcher usage. Currently we refetch the config from the http endpoint for restart, but not from the forge, making a unique interface pretty difficult.

[qwerty287]

I think it's fine to refetch in any case.

[anbraten]

I think it's fine to refetch in any case.

So not using the stored config at all and always use a fresh config from the forge / http endpoint?

[qwerty287]

Yes (config must still be stored to show in ui)

[anbraten]

PR should be finally ready now.

Adjusted the interface a bit again to include oldConfigData and a restart argument allowing me to handle it the same way we currently did. Could be get some refactoring and cleanup later on.

[codecov[bot]]

Codecov Report

Attention: 151 lines in your changes are missing coverage. Please review.

Comparison is base (e1521ef) 36.36% compared to head (f1133e0) 36.68%.

Files	Patch %	Lines
server/services/secret/db.go	33.82%	44 Missing and 1 partial :warning:
server/api/registry.go	0.00%	17 Missing :warning:
server/api/repo_secret.go	0.00%	17 Missing :warning:
server/api/global_secret.go	0.00%	11 Missing :warning:
server/api/org_secret.go	0.00%	11 Missing :warning:
server/pipeline/restart.go	0.00%	8 Missing :warning:
server/services/config/forge.go	92.00%	6 Missing and 2 partials :warning:
server/pipeline/items.go	0.00%	7 Missing :warning:
cmd/server/server.go	0.00%	5 Missing :warning:
server/pipeline/create.go	0.00%	4 Missing :warning:
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #915      +/-   ##
==========================================
+ Coverage   36.36%   36.68%   +0.32%     
==========================================
  Files         224      225       +1     
  Lines       14848    14782      -66     
==========================================
+ Hits         5399     5423      +24     
+ Misses       9057     8966      -91     
- Partials      392      393       +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[lafriks]

Did not test but code looks good