Item size has exceeded the maximum allowed size exception with ver 5.0.3

[sarbajitdutta]

Cannot store items more than 400kb using latest release of spring-data-dynamodb(5.0.3).

Expected Behavior

Store items more than 400kb.

Actual Behavior

We were using ver 4.3.1 of spring data dynamodb and was successfully able to store items which was more than 400kb. But updating to ver 5.0.3 getting Item size has exceeded the maximum allowed size

Steps to Reproduce the Problem

1. Clone https://github.com/wcmc-its/ReCiter.git
2. Setup local dynamoDB with PubmedArticle Table
3. Run http://localhost:5000/swagger-ui.html#/re-citer-controller/retrieveArticlesByUidUsingGET

Specifications

• Spring Data DynamoDB Version: 5.0.3 (2.0)
• Spring Data Version: 2.0.1.RELEASE
• AWS SDK Version:1.11.301
• Java Version:1.8.0_77 - Java HotSpot(TM) 64-Bit Server VM 25.77-b03
• Platform Details: Mac OS X 10.13.6 The model object we are trying to store in dynamodb table:

package reciter.database.dynamodb.model;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;

@DynamoDBTable(tableName = "PubMedArticle")
public class PubMedArticle {
    private Long pmid;
    private reciter.model.pubmed.PubMedArticle pubMedArticle;

    public PubMedArticle() {}

    public PubMedArticle(Long pmid, reciter.model.pubmed.PubMedArticle pubMedArticle) {
        this.pmid = pmid;
        this.pubMedArticle = pubMedArticle;
    }

    @DynamoDBHashKey(attributeName = "pmid")
    public Long getPmid() {
        return pmid;
    }

    @DynamoDBAttribute(attributeName = "pubmedarticle")
    public reciter.model.pubmed.PubMedArticle getPubmedArticle() {
        return pubMedArticle;
    }

    public void setPmid(Long pmid) {
        this.pmid = pmid;
    }

    public void setPubmedArticle(reciter.model.pubmed.PubMedArticle pubMedArticle) {
        this.pubMedArticle = pubMedArticle;
    }
}

The sample dynamodb json - 29547300.txt

[j-Halloran]

I could be wrong but I'm pretty sure this is an AWS limitation enforced by the newer DynamoDB sdk and not a choice that is up to @derjust .

See the dynamo db limits set by amazon here

[sarbajitdutta]

@derjust Does the library has support for storing binary type attributes? If we decide to compress the objects and call the save method with binary type (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/JavaDocumentAPIBinaryTypeExample.html) Also, is there any way we can use s3 to store the objects if its more than 400kb and have s3 location as a partition key for the table?