Count on secondary index causes missed key schema element

[peoplemerge]

I have a time series table UserActivity

case class UserActivity(
  id: Long,
  activity: String,
  occurred: Long = System.currentTimeMillis()
)
object UserActivity {
  val tableName = "UserActivityV2"
  object Attributes {
    val id = "Id"
    val activity = "Activity"
    val occurred = "Occurred"
  }
  val tableRequest =
    new CreateTableRequest()
      .withTableName(UserActivity.tableName)
      .withProvisionedThroughput(
        Schema.provisionedThroughput(10L, 5L))
      .withAttributeDefinitions(
        Schema.numberAttribute(Attributes.id),
        Schema.numberAttribute(Attributes.occurred),
        Schema.stringAttribute(Attributes.activity)
      )
      .withKeySchema(
        Schema.hashKey(Attributes.id),
        Schema.rangeKey(Attributes.occurred))
      .withGlobalSecondaryIndexes(
        new GlobalSecondaryIndex()
          .withIndexName("ActivitiesOccurred")
          .withKeySchema(
            Schema.hashKey(Attributes.activity),
            Schema.rangeKey(Attributes.occurred))
          .withProjection(new Projection().withProjectionType(ProjectionType.KEYS_ONLY))
          .withProvisionedThroughput(
            Schema.provisionedThroughput(10L, 5L)),

Using the AWS UI, I can query the index. Although I didn't find an example for running a query, I came up with:

mapper.countQuery[UserActivity](
      mapper.CountQueryMagnet.countQuerySecondaryIndex(
        "ActivitiesOccurred",
        "sometext",
        "Occurred",
        QueryCondition.greaterThan(1421023065979l)
      ))

However this blows up with:

com.amazonaws.AmazonServiceException: Query condition missed key schema element Activity (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: KD4364FBV747MPFBQALD9ONS17VV4KQNSO5AEMVJF66Q9ASUAAJG)
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1077)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:725)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:460)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:295)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3106)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.query(AmazonDynamoDBClient.java:1118)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBAsyncClient$18.call(AmazonDynamoDBAsyncClient.java:1557)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBAsyncClient$18.call(AmazonDynamoDBAsyncClient.java:1553)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Is this due to a feature that needs to be implemented upstream? See: https://github.com/aws/aws-sdk-ios/issues/75

My build.sbt has:

  "com.amazonaws" % "aws-java-sdk" % "1.9.14"

that's latest, at least as of last week.

UPDATE: Here is the serializer

  implicit object userActivitySerializer
    extends DynamoDBSerializer[UserActivity] {

    override val tableName = UserActivity.tableName
    override val hashAttributeName = Attributes.id
    override val rangeAttributeName = Some[String](Attributes.occurred)

    override def primaryKeyOf(userActivity: UserActivity) =
      Map(Attributes.id -> userActivity.id,
        Attributes.occurred -> userActivity.occurred
      )

    override def toAttributeMap(userActivity: UserActivity) =
      Map(
        Attributes.id     -> userActivity.id,
        Attributes.email -> userActivity.email,
        Attributes.activity  -> userActivity.activity,
        Attributes.occurred -> userActivity.occurred
      )

    override def fromAttributeMap(
      item: collection.mutable.Map[String, AttributeValue]) =
      UserActivity(
        id     = item(Attributes.id),
        email = item(Attributes.email),
        activity  = item(Attributes.activity),
        occurred = item(Attributes.occurred)
      )
  }

Hmm, looks like there may be a way to identify keys there. That could be my problem.

[peoplemerge]

I think I just made some progress!

https://github.com/pellucidanalytics/aws-wrap/blob/master/src/main/scala/dynamodb/mapper.scala#L1203

Clearly that's looking at the serializer's hash/range pair. I currently have only a single serializer, which is for the primary key, and hardcoding hashAttributeName and rangeAttributeName works! Is the intent here that with (potentially multiple) GSIs, should I be making a serializer for each, perhaps extending from a base? Or is implementation currently LSI-only?

[dwhjames]

Hi @peoplemerge, I’m sorry I missed this.

Did you find a resolution? Let me know if there is anything I can still do to help.

[mingchuno]

@peoplemerge I have forked this repo and published it under version 0.9.0 https://github.com/mingchuno/aws-wrap

Can you please check if you still have any problems? If yes, please submit an issue there