boto3 icon indicating copy to clipboard operation
boto3 copied to clipboard

Glue client, Get get_tables method fails

Open parshnt opened this issue 3 years ago • 4 comments

Describe the bug

While calling Glue client.get_tables, the method fails when the size of the Expression passed in params is greater than 255 chars

complete error:

ERROR An error occurred (ValidationException) when calling the GetTables operation: 1 validation error detected: Value 'big_value' at 'expression' failed to satisfy constraint: Member must have length less than or equal to 255

Expected Behavior

this size should be 2048 characters, as mentioned on the glue API docs page:

https://docs.aws.amazon.com/glue/latest/webapi/API_GetTables.html#API_GetTables_RequestSyntax

Screenshot 2022-04-18 at 6 13 45 PM the

Current Behavior

currently, the SDK throws an error when the limit of expression goes beyond 255 chars,

Reproduction Steps

attaching the code snipped to reproduce:

`params = { "DatabaseName": "db_name", "MaxResults": 100, "CatalogId": "catalog_id", "Expression": "abc|abc2|.....more than 255 chars" }

client = boto3.client("glue") response = client.get_tables(**params)`

Possible Solution

Make the limit 2048 chars (as specified according to spec)

Additional Information/Context

No response

SDK version used

1.11.17

Environment details (OS name and version, etc.)

python:3.8-slim-buster Docker image

parshnt avatar Apr 18 '22 13:04 parshnt

Hi @parshnt,

This looks like a server-side validation error. Can you please provide the output with debugging turned on? You can add boto3.set_stream_logger('') to your code to do this. Please be sure to redact any sensitive information.

Thanks!

kdaily avatar Apr 18 '22 17:04 kdaily

@kdaily attaching the debug logs

on that note some things I've observed:

  • This happens only when the job runs on our EKS cluster, the local system seems to be working fine
  • I've updated the boto3, botocore to the latest available versions (at this time) and the issue still persists.

glue-get-tables-debug.log

I've attached a subset of logs (relevant to the issue, w redacted info). let me know if anything else is needed (can share over a secure channel w you).

parshnt avatar Apr 21 '22 13:04 parshnt

@kdaily have you been able to validate or replicate the issue?

parshnt avatar Apr 28 '22 08:04 parshnt

Hi @parshnt I tried to reproduce this but wasn't able to. Instead I got the following error once exceeding 2048 characters: '...' at 'expression' failed to satisfy constraint: Member must have length less than or equal to 2048

If you update to the latest version of boto3 do you still see this issue?

tim-finnigan avatar Jul 07 '22 17:07 tim-finnigan

Since we haven't heard back in a few weeks I'm going to close this issue. Please let us know if you're still running into this issue and we can revisit it. Thanks!

tim-finnigan avatar Sep 15 '22 17:09 tim-finnigan