ItemUtils in SDK V2

[reply2srij]

Please bring back something similar to ItemUtils from sdk v1 to sdk v2. I checked #1975 which was closed due to non-feasibility. But, this would mean all the developers would need to implement the utility independently, which is not ideal. Opening this as feature request again so it can gain attention.

[debora-ito]

We'll take the ask into consideration and reevaluate if we'd like to add support for a ItemUtils equivalent.

[davpow]

Please bring back the ItemUtils in SDK V2. It's very difficult to work with the dynamodb items without this helper class. Thanks.

[juuusto]

Please bring this back!

[debora-ito]

Hey everyone, this is a Request for Comments:

Since ItemUtils.java is fairly extensive utility and there was a change in design from v1 DynamoDB Mapper to v2 DynamoDB Enhanced Client, to help us understand the use cases of a v2 ItemUtils equivalent please comment here which utility methods you'd like it to have, which methods you'd use the most. Be as specific as you can.

[valfirst]

@debora-ito in our case we heavily rely on

https://github.com/aws/aws-sdk-java/blob/f1516cb5beefc00a39fa1b7a884e574289d457bd/aws-java-sdk-dynamodb/src/main/java/com/amazonaws/services/dynamodbv2/document/ItemUtils.java#L49-L54

as an entry point

[reply2srij]

To and From conversion between attributeValueMap and regular JSON string:

Map<String, AttributeValue> jsonStringToAttrValueMap (String contentJsonString); String attributeValueMapToJsonString(Map<String, AttributeValue> map);

[dscalzi]

Converting to and from a Map<String, AttributeValue> to a Map<String, Object>. The latter being a required format for processing with Jackson/gson.

As an example, we want to be able to pull the raw object out of DynamoDB, run some processing on it, and convert it into a simple map (Map<String, Object>) for whatever post-processing. This could be deserialization into a POJO or just passing it to further map-based utilities. There are use cases where the DynamoDB mapper is not suitable, or adds extensive overhead. Having the ability to bypass this and use utilities to "map manually" is really critical.

Even in the DynamoDB console, there is a direct option to toggle between a simple map and the raw (attribute value) map. I'm not sure why this functionality isnt included in the SDK v2. Like I said in the other thread, it is really a degradation.

I'm also a user of the JavaScript SDK v3, what's really needed is a direct equivalent of marshall/unmarshall (same thing as ItemUtils in v1).

const { marshall, unmarshall } = require("@aws-sdk/util-dynamodb");

ref: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/modules/_aws_sdk_util_dynamodb.html

[marko-asplund]

@debora-ito Any news on this? Are the use cases clear?

[evankozliner]

Honestly very surprised they shipped v2 without these

Please bring back itemutils.valToString!

[shihuayong]

Please bring it back!!! It takes a long time for me to find the utils. What I found is this OPEN issue :(

[refacktor]

It is mind-boggling that a "NoSQL" database in 2022 cannot store and retrieve plain JSON. And that is exactly the case when using DynamoDB with Java and sdk v2. Some of us old-timers know about sdk v1 and ItemUtils, but I expect newer developers will just move on and adopt a competing alternative.

[bluecapz]

Pitching my use case if it isn't clearly stated from previous posts. Basically when supporting paginated queries & such where there will be involvement of lastEvaluatedKey or exclusiveStartKey.

Current API only accepts a Map<String, AttributeValue> which is not a problem when paginated calls occur within the service itself. But when the service is a RESTful backend that needs to return this key, it will eventually be returned to caller as a String and the next time they will pass it, as a String as well.

It really is a chore to manually write utility classes to handle this. So a native SDK support for JSON to Map<String, AttributeValue> and vice versa will be valuable, particularly for RESTful services with pagination support

[clant-eb]

Hi, curious to know if there were any updates to this? I'm in need of converting attributeValueMap to a regular JSON string 😓

[mhepburn]

How is this still an unresolved issue? Every other language can do this easily.

[sanathkumarx]

Check - https://github.com/aws/aws-sdk-java-v2/issues/2151

https://www.davidagood.com/dynamodb-enhanced-client-java-heterogeneous-item-collections/

In short:

import software.amazon.awssdk.enhanced.dynamodb.TableSchema;

TableSchema.fromClass(Class<T>);
Map<String, AttributeValue> image;
T item = tableSchema.mapToItem(image);

[joviegas]

We have added EnhancedDocument which features APIs such as fromJson, fromAttributeValueMap, toJson, and toMap. This covers nearly all of the type-safe APIs found in ItemUtils.

Aws sdk java 2.x does not have any plans to introduce APIs that lack type safety as compared to ItemUtils. If you come across a use case requiring additional type-safe APIs not already covered in EnhancedDocument, please provide a use case for consideration.

[joviegas]

Closing the issue after monitoring the above comment for 3 months. Please refer above comment.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[theon]

I tried using EnhancedDocument.toJson, but it didn't give me quite the result I wanted (not sure if I missed something). I was expecting to get the JSON with the explicit DynamoDB types, e.g:

{
  "M": {
    "MyStringField": { "S": "MyString" }
  }
}

But was getting the concise JSON, e.g:

{
  "MyStringField": "MyString"
}

I ended up using this. Which is kinda horrible, but gets the json we need and we were only using it in tests so 🤷 It's in Scala, but should be easy to translate to the equivalent Java:

Turn AttributeValue av into a Json String:

    val marshaller = JsonProtocolMarshallerBuilder
      .create()
      .operationInfo(OperationInfo.builder().httpMethod(SdkHttpMethod.PUT).build())
      .endpoint(java.net.URI.create("http://dummy"))
      .protocolMetadata(
        AwsJsonProtocolMetadata.builder().protocol(AwsJsonProtocol.AWS_JSON).build()
      )
      .jsonGenerator(
        AwsStructuredPlainJsonFactory.SDK_JSON_FACTORY.createWriter("application/json")
      )
      .build()
    val request = marshaller.marshall(av)
    val bytes   = request.contentStreamProvider().get().newStream().readAllBytes();
    new String(bytes, StandardCharsets.UTF_8)

Parse String jsonString into AttributeValue:

    val u = JsonProtocolUnmarshaller
      .builder()
      .defaultTimestampFormats(Map.empty.asJava)
      .parser(JsonNodeParser.create())
      .build()
    val av = AttributeValue.builder()
    u.unmarshall(
      av,
      SdkHttpFullResponse
        .builder()
        .content(
          AbortableInputStream.create(new StringInputStream(jsonString))
        )
        .build()
    )
    av.build()