RedshiftDataAPIService saying parameters are not required but error is saying they are

[seanyboi]

Describe the issue

When using the RedshiftDataAPIService execute_statement, it states that only the database name and sql is required as shown here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/redshift-data.html#RedshiftDataAPIService.Client.execute_statement

But then when used in python I receive the error stating:

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ExecuteStatement operation: Either ClusterIdentifier or WorkgroupName needs to be specified.

So what is correct?

Links

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/redshift-data.html#RedshiftDataAPIService.Client.execute_statement

[nateprewitt]

Hi @seanyboi, either the ClusterIdentifier or WorkgroupName can be conditionally required for certain workloads. The docs page currently defines the cases where they might be needed.

ClusterIdentifier (string) -- The cluster identifier. This parameter is required when connecting to a cluster and authenticating using either Secrets Manager or temporary credentials.

WorkgroupName (string) -- The serverless workgroup name. This parameter is required when connecting to a serverless workgroup and authenticating using either Secrets Manager or temporary credentials.

It sounds like your workflow falls under one of these cases. Was there confusion on which parameter to use with your current invocation?

[seanyboi]

@nateprewitt yeah I realised I needed a WorkgroupName. I guess i'm saying it wasn't really clear if something is conditionally required? Maybe something in the docs needs to be placed to make it known.

Something that was also strange is when I used the RedshiftDataAPIService on boto3 a month ago neither ClusterIdentifier or WorkgroupName were needed? Are you able to shine some light on this maybe?

Thanks for the response!

[nateprewitt]

I guess i'm saying it wasn't really clear if something is conditionally required?

This is unfortunately a representation issue for dynamic cases where the requirement is exchangeable between multiple values. We could reach out to the service team to perhaps raise this higher in the operation in case the docs are not fully read.

Something that was also strange is when I used the RedshiftDataAPIService on boto3 a month ago neither ClusterIdentifier or WorkgroupName were needed?

Could you provide a bit more detail on this. What version of Boto3 were you using when this worked and has anything else around your cluster/authentication setup changed in that time? ClusterIdentifier has been present and conditionally required for almost a year. The WorkgroupName is part of a recent feature which allows execute to be run on a serverless workgroup which would require an opt-in to enable.

[seanyboi]

I guess i'm saying it wasn't really clear if something is conditionally required?

When using boto3 I normally go straight to the parameters to see what's needed and the [Required] part really helps me make my development more efficient and speedy so I only found out when I tried running it and the code error'd out.

The WorkgroupName is part of a recent feature which allows execute to be run on a serverless workgroup which would require an opt-in to enable.

boto3 v1.24.0 has been used on my side for 3+ months, as I mentioned a month ago I never needed any of those two parameters. The only key difference I've noticed is this within the redshift console the status of the WorkgroupName says it has failed? It seems to be a breaking change feature in my eyes, without a change of lib version.

I see someone else has mentioned this on re:Post too here: https://repost.aws/questions/QUCOHIJLh6RDO3yTOCFtAC8A/how-do-i-solve-a-failed-message-on-a-redshift-serverless-workgroup

[nateprewitt]

Thanks for the reply, @seanyboi. To clarify, does 1.24.0 still work, or is it also broken now? Do you know roughly when you started receiving this error? I can follow up with the service team for clarification on the change, but I do agree this seems incorrect if the execution previously didn't require a WorkgroupName.

[seanyboi]

1.24.0 works if I provide it with a WorkgroupName, my companies repo is quite lightweight so I was able to make the changes to accommodate this, but could affect anyone bigger if they rely on cron jobs/tasks etc. I only noticed the error this morning (United Kingdom time) when trying to use the RedshiftDataAPIService for a different use case :)

[tim-finnigan]

Checking in - as previously mentioned here the service team is the owner of their API/documentation so any issues related to that should be redirected to them. You could send feedback using the Provide feedback link at the bottom of the API documentation page or reach out through AWS Support for further escalation.

Alternatively we can forward issues to service teams on your behalf. If that's how you'd like to proceed can you please let us know what the current status of this issue is and what you'd like to report to the service team?