Documentation error in DynamoDB.Table class

[rwstan]

The Table class has Actions that simplify the interface and response structure. In particular get_item, put_item and update_item use the following parameter:

Key={
  'string': 'string'|123|Binary(b'bytes')|True|None|set(['string'])|set([123])|set([Binary(b'bytes')])|[]|{}
},

rather than, as for the corresponding methods supported by the DynamoDB.Client class,

Key={
  'string': {
      'S': 'string',
      'N': 'string',
      'B': b'bytes',
      'SS': [
          'string',
      ],
      'NS': [
          'string',
      ],
      'BS': [
          b'bytes',
      ],
      'M': {
          'string': {'... recursive ...'}
      },
      'L': [
          {'... recursive ...'},
      ],
      'NULL': True|False,
      'BOOL': True|False
}

However the documentation says that both take AttributeValue objects:

Key (dict) -- [REQUIRED]

A map of attribute names to AttributeValue objects, representing the primary key of the item to retrieve.

Similarly for the response structure:

Item (dict) --

A map of attribute names to AttributeValue objects, as specified by ProjectionExpression.

This seems to be misleading. In the case of the actions of the Table class, the parameter is a map of attribute names to 'raw' attributes, not AttributeValue objects, and so on. I think all references to AttributeValue objects in the documentation for the Table class need to be reviewed.

[tim-finnigan]

Hi @rwstan thanks for reaching out. Our service API documentation is generated upstream. For example the get_item documentation comes from here: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html

There is a Provide feedback link on that or the AttributeValue page if you want to give feedback to the DynamoDB team regarding the wording or descriptions. Please let me know if you had any follow up questions.

[rwstan]

Hi @tim-finnigan, it looks to me like Service Resources in boto3 are defined by boto3 and not the underling API. (From the docs: "Resources represent an object-oriented interface to Amazon Web Services (AWS). They provide a higher-level abstraction than the raw, low-level calls made by service clients."). The issue is how you are documenting the DynamoDb Service Resource and its sub-resource Table. I can't see how the documentation for that is generated upsteam?

[tim-finnigan]

Hi @rwstan here is the documentation page for DynamoDB Service Resources: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#service-resource

The documentation page I linked here is pulled into the documentation for other SDKs (for example JavaScript). Was there another documentation page that you were referring to originally? Please share the link to it if so.

[rwstan]

Sorry, let me try and be a bit clearer.

There is an error here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.get_item

"Key (dict) -- [REQUIRED]

A map of attribute names to AttributeValue objects, representing the primary key of the item to retrieve."

This isn't right, because it's a map of attribute names to 'pure' values, e.g. a string or a number, not an AttributeValue object.

Corresponding errors appear elsewhere in the documentation for the DynamoDB Table service sub-resource (not the DynamoDB client).

Are you saying there is upstream documentation for the boto3 DynamoDB service resource, where these errors originate? Because these seem to be language-specific (i.e. specific to boto3 and different to the JavaScript SDK).

[tim-finnigan]

The documentation you quoted comes from the DynamoDB service team. The AttributeValue is documented here:

Represents the data for an attribute.

Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

So an "AttributeValue object" can be a string, number, or other data type documented there.

[rwstan]

Hi @tim-finnigan

I don't think you're seeing the problem yet.

Here are two ways of doing the same thing. Suppose I have a table called FruitTable with partition key id of type Number and no sort key. I have an item with the attributes id: 1 and FruitType: Apple.

First, via the DynamoDB client

db_client = boto3.client('dynamodb')
response = db_client.get_item(
    TableName='FruitTable',
    Key={
        'id': {
            'N': '1'
        }
    }
)
print(response['Item'])
# outputs {'id': {'N': '1'}, 'FruitType': {'S': 'Apple'}}

As you can see, I have to provide the Key as a map from the attribute name id to an AttributeValue object that makes the type (N) explicit. Similarly the response maps attribute names to AttributeValue objects. All this is well documented.

Second, via the DynamoDB resource

db_resource = boto3.resource('dynamodb')
table = db_resource.Table('FruitTable')
response = table.get_item(
    Key={
        'id': 1
    }
)
print(response['Item'])
# outputs {'id': Decimal(1), 'FruitType': 'Apple'}

In this case, I provide the Key as a map from the attribute name id to (simply) an integer. Similarly the response maps attribute names to simple types, an integer and a string, not AttributeValue objects.

However, the documentation for the resource says (my emphasis):

Key (dict) -- [REQUIRED]

A map of attribute names to AttributeValue objects, representing the primary key of the item to retrieve.

This same error is repeated in the docs describing the response, and similarly for put_item and query. I can see lots of instances of the term AttributeValue in the documentation of the service resource that I suspect are wrong (it should be attribute value, not AttributeValue object).

Having dug around a bit, I can see that the transformation between AttributeValue objects and simple attribute values appears to be done here. That file even has functions to transform the docs accordingly:.

# Apply the documentation customizations to account for
# the transformations.
attr_value_shape_docs = DocumentModifiedShape(...)

My claim is that these document customizations don't do a complete job, i.e. that there are bits of the docs left not transformed which leads to confusion and ambiguity.

[tim-finnigan]

Thanks @rwstan for following up, I see what you’re saying now. I will set this as a feature request to improve the resource documentation.

[martinth]

The same issue is also present on the documentation for scan. If your read the docs you will come to the conclusion that you are supposed to call it like this:

client.scan(
      TableName=TABLE_NAME,
      FilterExpression='#pid = :pid',
      ExpressionAttributeValues={
          ':pid': {
              'S': pid
          }
      },
      ExpressionAttributeNames={
          "#pid": "pid_column"
      }
  )

Where you are actually are supposed to call it like

    r = client.scan(
        TableName=TABLE_NAME,
        FilterExpression='#pid = :pid',
        ExpressionAttributeValues={
            ':pid': pid
        },
        ExpressionAttributeNames={
            "#pid": "pid_column"
        }
    )

I wasted half a day reducing down my code and looking for the issue until I finally took the great fun of settings breakpoints very deep in boto3 and saw that it was actually sending an API call with this crap of a body:

b'{"TableName": "the_table", "FilterExpression": "#pid = :pid", "ExpressionAttributeValues": {":pid": {"M": {"S": {"S": "some_value"}}}}, "ExpressionAttributeNames": {"#pid": "pid_column"}}'

The worst part of this is not that the documentation is wrong but that it has been wrong for one and a half years and that there is no error message at all (Because why would it? I know this is a perfectly reasonable DynamoDB call).

I don't think I have ever been so mad at a piece of software!