SQS service controller

[tabern]

New ACK service controller

Support for Amazon SQS.

List of API resources

List the API resources in order of importance to you:

• Queue

[ack-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

[RedbackThomson]

/lifecycle frozen

[sergey-telpuk]

Hello, any updates?

[jaypipes]

Apologies @sergey-telpuk, but SQS has lagged behind :( I hope to get back to working on it this quarter.

[sergey-telpuk]

@jaypipes Hello, thanks for answer. I really want to start using SQS with help of this operator. I'll be waiting for work done

[paulermo]

@jaypipes we will wait too, hope things will go well and coding wil be fast:)

[satdeveloping]

@jaypipes Do you have a roadmap/list of issues that the community can work on? A lot of us are looking for a well supported SQS operator.

[RedbackThomson]

@jaypipes Do you have a roadmap/list of issues that the community can work on? A lot of us are looking for a well supported SQS operator.

The SQS controller needs to be updated in the form of:

• Updating the runtime from a very old version (v0.0.6) to the latest version, and re-generating to fit that runtime
• Updating to Go v1.17
• Adding tests for the Queue resource

The hardest part of this will be re-generating, as our generator configuration has changed a lot since v0.0.6, so you may need to make changes to the generator.yaml to fill in some of the gaps.

[jaypipes]

Unfortunately, the code generator needs to be refreshed to work with attribute-based field definitions like SNS and SQS. This refresh work is not easy and I need to find time to get it done in the next couple sprint cycles as Bryan works on refactoring other parts of the code generator.

[dlgordon]

Hello

I spent some time this weekend having a look at the SQS controller and made some progress - interested if a) I'm on the right track and b) this is useful effort. I know SQS is in progress as per the roadmap.

Anyway, I was able to upgrade the SQS Controller to the latest packages, update the generator yaml (https://github.com/dlgordon/sqs-controller/blob/update-sqs-controller/generator.yaml) and generate a new controller which works in the Create and Delete cases. Far as I can see, my branch mostly works with the following exceptions

• Creating a queue fails if you don't set any attributes (https://github.com/dlgordon/sqs-controller/blob/5fc6684e215c8cf1848095564577e69143575d14/pkg/resource/queue/sdk.go#L204) due to an AWS API error. Just need to add a check there (I think the SNS controller does this)
• Editing/updating the queue attributes doesn't work because sdkFind doesn't set the ko.spec fields based on the returned attribtes (https://github.com/dlgordon/sqs-controller/blob/5fc6684e215c8cf1848095564577e69143575d14/pkg/resource/queue/sdk.go#L81-L90) - I can probably fix with a custom template injection, or perhaps by an update to the field handler to deal with the attributes correctly (which would then work for SNS as well)
• Obviously no e2e tests yet

I hope this is of interest and am keen for any feedback - like I said, in my superficial testing, my branch works well enough for creating and deleting queues but I'm happy to take a steer if I am way off course.

[jaypipes]

@dlgordon you rock. thank you for taking the initiative here and making some progresson the SQS controller. I'd be more than happy to review a PR from you and collaborate on improvements to the SQS controller.

[dlgordon]

Thanks, I appreciate that - I have looked further and think I will approach the next steps like this -

• I'll add some e2e tests for Create and Delete of Queue's and then work through the PR process for the SQS controller at that stage

• To fix the sdkFind issue, instead of using a custom template, I think it would be best to fix up SetResourceGetAttributes in the Generator (pkg/generate/code/set_resource.go), which would have the benefit of then working for SNS as well. Same goes for the check on create when calling SetAttributes

So that will be 2 separate PR's to start with if that work. I can probably get the e2e for SQS create/delete done in a couple of days, so that will be first. At least that PR will get the generator packages etc current. Then will move onto making those proposed generator changes for the Attributes based APIs

[bwhaley]

Based on the latest updates here, should the Project Stage on the SQS controller be updated to PLANNED?

[RedbackThomson]

@bwhaley That documentation page is automatically generated based on the status of the Github issue, the repository release numbers and the existence of the stable Helm chart. Since a Github issue exists, but no release has been created for that repository, the SQS controller is still considered Planned. Once we publish the first release, it should update into Released status

[jaypipes]

Based on the latest updates here, should the Project Stage on the SQS controller be updated to PLANNED?

Yeah... it actually should be in IN PROGRESS project stage. That listing is generated with code here:

https://github.com/aws-controllers-k8s/test-infra/blob/a69375ed29089c5bd48c2ae99e12b9ee93d11fbb/tools/ackdiscover/controller.py#L211

Where is looks for whether the service's GH issue is in the Planned column of the service controller project. I need to add a block in there to look for projects in the In Progress column....

[bwhaley]

Thanks for the explanation, folks! I just wanted to get a pulse on it. Looking forward to this one.

[jaypipes]

@bwhaley @dlgordon @sergey-telpuk apologies for the long delay in working on the SQS controller. Recently, I've made some progress in getting the attribute-based APIs modernized to the latest ACK code generator. I'm hoping to get both SNS and SQS into solid shape by end of the year.

[niwczi]

Hi there! Any news regarding the SQS controller? IHAC that is looking for such a controller. Happy holidays!

[jaypipes]

Hi @niwczi! We released an updated version of the SQS controller 10 days ago (v0.0.1). You can read about the Queue resource here:

https://aws-controllers-k8s.github.io/community/reference/sqs/v1alpha1/queue/

We're heading towards another release that will include update support soon and then GA in early 2023.