components-contrib icon indicating copy to clipboard operation
components-contrib copied to clipboard

refactored aws sqs policy inserting

Open xingrux opened this issue 3 years ago • 11 comments

Signed-off-by: Xingru [email protected]

Description

For aws sqs pubsub component, when adding a new statement, add it to an array of sns arns instead of add a new statement to avoid over limit 20, which is the maximum number of access policy statement

Issue reference

This pr fixes #1780

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • [x] Code compiles correctly
  • [x] Created/updated tests
  • [ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

xingrux avatar Jun 21 '22 15:06 xingrux

Looks good to me, my only question is what happens to existing statements (i.e. during an upgrade). Will this cause any issue?

ItalyPaleAle avatar Jun 21 '22 15:06 ItalyPaleAle

@amimimor can you please review this?

yaron2 avatar Jun 21 '22 18:06 yaron2

FYI - this will not be part of release 1.8

berndverst avatar Jun 21 '22 23:06 berndverst

Looks good to me, my only question is what happens to existing statements (i.e. during an upgrade). Will this cause any issue?

Thanks for reviewing! From my understanding, the statements will be overwritten by SetQueueAttributesWithContext. And in practice, I have a custom access policy document and every time I deploy, it is overwritten by dapr, and I replace it with my custom one.

xingrux avatar Jun 22 '22 07:06 xingrux

FYI - this will not be part of release 1.8

Thanks for the information. Is there any way to build this branch myself and use it?

xingrux avatar Jun 22 '22 07:06 xingrux

FYI - this will not be part of release 1.8

Thanks for the information. Is there any way to build this branch myself and use it?

Absolutely.

First you need to clone github.com/dapr/dapr, then you need to import your branch of components-contrib instead of what is there.

You can do this via: go get https://github.com/xingrux/components-contrib@master

Next you will want to make sure you run go mod tidy - for convenience I recommend running this via make modtidy-all.

Now you are ready to build your own version of Dapr: Take a look at the step by step option documented here https://github.com/dapr/dapr/blob/master/tests/docs/running-e2e-test.md#option-2-step-by-step-guide

You can skip the step for running the E2E test apps. Instead you can just deploy your applications however you want.

Oh and if you don't use Kubernetes it is event easier, just simply run make build.

berndverst avatar Jun 22 '22 20:06 berndverst

First you need to clone github.com/dapr/dapr, then you need to import your branch of components-contrib instead of what is there.

You can do this via: go get https://github.com/xingrux/components-contrib@master

Next you will want to make sure you run go mod tidy - for convenience I recommend running this via make modtidy-all.

Now you are ready to build your own version of Dapr:

Thank you very much. I’ve test the build in practical environment and found some bugs. I’ve fixed them this week and will push the changes this weekend.

xingrux avatar Jul 01 '22 21:07 xingrux

@xingrux please fix the linter issues and we'll merge this PR:

Error: File is not `gofumpt`-ed (gofumpt)
Error: File is not `gofumpt`-ed (gofumpt)
Error: File is not `goimports`-ed with -local github.com/dapr/ (goimports)

yaron2 avatar Jul 27 '22 16:07 yaron2

@xingrux fix linter errors and we can merge: https://github.com/dapr/components-contrib/runs/7729233136?check_suite_focus=true.

yaron2 avatar Aug 08 '22 15:08 yaron2

See current errors:

  Running [/home/runner/golangci-lint-1.45.2-linux-amd64/golangci-lint run --out-format=github-actions --timeout 15m] in [] ...
  Error: expected '(', found Test_buildARN_DefaultPartition (typecheck)
  Error: expected '(', found Test_buildARN_StandardPartition (typecheck)
  Error: expected '(', found Test_buildARN_NonStandardPartition (typecheck)
  Error: expected '}', found 'EOF' (typecheck)

yaron2 avatar Aug 09 '22 20:08 yaron2

Ping @xingrux

yaron2 avatar Aug 12 '22 20:08 yaron2

ping @xingrux

yaron2 avatar Aug 15 '22 22:08 yaron2

@yaron2 Thanks! I've fixed the error.

xingrux avatar Aug 20 '22 19:08 xingrux

Codecov Report

Merging #1807 (b52be2c) into master (9c9df2f) will increase coverage by 0.05%. The diff coverage is 60.86%.

@@            Coverage Diff             @@
##           master    #1807      +/-   ##
==========================================
+ Coverage   37.82%   37.87%   +0.05%     
==========================================
  Files         192      192              
  Lines       24094    24113      +19     
==========================================
+ Hits         9114     9134      +20     
+ Misses      14209    14207       -2     
- Partials      771      772       +1     
Impacted Files Coverage Δ
pubsub/aws/snssqs/snssqs.go 5.00% <0.00%> (+0.09%) :arrow_up:
pubsub/aws/snssqs/policy.go 65.90% <65.11%> (+65.90%) :arrow_up:
state/in-memory/in_memory.go 42.58% <0.00%> (-3.43%) :arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Aug 22 '22 20:08 codecov[bot]