community icon indicating copy to clipboard operation
community copied to clipboard

RDS aurora postgresql - doc says allocatedStorage isn't applicable, resource status say must be provided, and fails if provided

Open rgarrigue opened this issue 3 years ago • 12 comments

Describe the bug

The doc state you shouldn't provide allocatedStorage for aurora. But when done as recommended it fails

Message:               InvalidParameterValue: The parameter AllocatedStorage must be provided and must not be null.

If I specified one nonetheless

Message:               InvalidParameterCombination: Invalid iops to storage (GB) ratio for engine name aurora-postgresql and storage type aurora: 0.0000

So I tried with IOPS, why not

Message:               InvalidParameterCombination: Specifying IOPs is not allowed for this engine

Checkmate.

Steps to reproduce

Something like that, but I believe there's no valid combination allowing to create an aurora postgresql anyway

apiVersion: rds.services.k8s.aws/v1alpha1
      kind: DBInstance
      metadata:
        name: "strapi-cms-database-instance"
      spec:
        allocatedStorage: 10 # Not applicable to Aurora
        dbInstanceClass: db.t4g.small
        dbInstanceIdentifier: "test"
        engine: aurora-postgresql
        engineVersion: "14"
        # iops: 50 # Not applicable to Aurora
        ...

Expected outcome

As the doc says : no allocatedStorage, no IOPS

Environment

EKS 1.23, ACK RDS 0.1.0

rgarrigue avatar Sep 14 '22 12:09 rgarrigue

:facepalm:

Gosh ... DBCluster exist, and I've been trying with DBInstance ...

I guess, forget the above. But rip off any mention of aurora in the DBInstance doc -___-

rgarrigue avatar Sep 14 '22 12:09 rgarrigue

@rgarrigue Glad you were able to figure this out!

The Amazon RDS API docs do state that AllocatedStorage is not applicable for Aurora: https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstance.html. This information is also in the docs ACK provides for DBInstance The Amazon RDS tutorial also does not use Amazon Aurora for any of the initial creation examples, though it has an example for doing a create from snapshot.

It is possible to use DBInstance for Amazon Aurora as you can add additional instances to the DBCluster.

If you can make a specific suggestion to where the docs can be clearer, please let me know! Otherwise, I do not think there is a bug here.

jkatz avatar Sep 14 '22 14:09 jkatz

About the documentation question, I've one : add examples.

Terraform does this reaaaally well. Here's a random example from a resource I didn't even paid attention what it is, but I know I can get something running straight away https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appflow_connector_profile. And I know there are examples there for all the scenarios. Another example, the terraform FOSS modules often have those FOSS modules https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples. Both are a good part of the reason I appreciate Terraform so much : I can pin my TF / module version, use the example and have a reliable result.

I'll let you keep / close this issue if you want it to keep alive the doc topic or not.

rgarrigue avatar Sep 16 '22 07:09 rgarrigue

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

ack-bot avatar Mar 14 '23 07:03 ack-bot

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle rotten

ack-bot avatar Apr 13 '23 08:04 ack-bot

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

ack-bot avatar Jul 12 '23 15:07 ack-bot

Stale issues rot after 60d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 60d of inactivity. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle rotten

ack-bot avatar Sep 10 '23 17:09 ack-bot

The same issue with DBInstance. I am trying naive approach to create Aurora DB instance. So if I omit allocatedStorage, I am getting the error

The parameter AllocatedStorage must be provided and must not be null

but if I am explicitly set it:

    - message: "InvalidParameterCombination: Invalid storage size for engine name aurora-mysql and storage type aurora: 20\n\tstatus code: 400, request id: 47e7b868-2952-4865-a71f-26a8ecd1a16c"
      status: 'True'
      type: ACK.Terminal

So I am kindly asking to provide some nice and working templates for creating of Aurora instances of different types.

Also I suppose that it is possible that some bug in API exists (like https://github.com/aws/aws-sdk-go/issues/3222 )

gecube avatar Oct 03 '23 09:10 gecube

I becoming nervous :-) Colleagues, did anybody succeed with Aurora DB Instance creation?

Test case:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBInstance
metadata:
  name: artemis-uat
  namespace: infra-uat
spec:
  allocatedStorage: 40
  dbInstanceClass: db.r5.large
  dbInstanceIdentifier: artemis-uat
  engine: aurora-postgresql
  engineVersion: "15.3"
  masterUsername: "root"
  masterUserPassword:
    namespace: infra-uat
    name: artemis-db-password
    key: password
  dbSubnetGroupRef:
    from:
      name: rds-subnet
  publiclyAccessible: false

yields

    - message: "InvalidParameterValue: The parameter AllocatedStorage must be provided and must not be null.\n\tstatus code: 400, request id: 584fb457-ebb9-48ce-9761-43ffcda61f7b"
      status: 'True'
      type: ACK.Terminal

If I add allocatedStorage: 40:

    - message: "InvalidParameterCombination: Invalid iops to storage (GB) ratio for engine name aurora-postgresql and storage type aurora: 0.0000\n\tstatus code: 400, request id: 0ebc7978-f36e-4a58-a7ec-6fc3ea7afc0e"
      status: 'True'
      type: ACK.Terminal

gecube avatar Oct 04 '23 08:10 gecube

I am also seeing the same error with aurora-postgresql with the same YAML as previous commenters with chart version 1.1.8.

InvalidParameterCombination: Invalid iops to storage (GB) ratio for engine name aurora-postgresql and storage type aurora: 0.0000\

vsimon avatar Dec 19 '23 19:12 vsimon

Issues go stale after 180d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 60d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

ack-bot avatar Jun 16 '24 19:06 ack-bot

/remove-lifecycle stale

gecube avatar Jun 16 '24 19:06 gecube