Dynamoid icon indicating copy to clipboard operation
Dynamoid copied to clipboard

Support for ActiveRecord "subset conditions"

Open lisad opened this issue 12 years ago • 2 comments

ActiveRecord supports "subset conditions", e.g. http://edgeguides.rubyonrails.org/active_record_querying.html has the example:

Client.where(orders_count: [1,3,5])

DynamoDB supports "BatchGet" : http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/DynamoDB/BatchGet.html

It would seem to be a good matchup. Dynamoid does not currently support this. When I attempt the DynamoidThing.where(id: array_of_ids) syntax, it fails in Dynamoid::Adapter::AwsSdk.query, which is documented to only support one hash key.

If this is a good fit and Dynamoid should be extended to support it, any suggestions how to integrate it? I would propose either in Dynamoid::Adapter.query or Dynamoid::Adapter::AwsSdk.query, detect the Array passed in and branch off the logic. Then in Dynamoid::Adapter::AwsSdk, create a new "batch_query" method that uses batch_get something like this (untested, just a sketch)

  def batch_query(table_name, hash_keys)
      items = hash_keys.map { |key| Item.new(:hash_value => key) }
   batch_get.table(table_name, :all, items)
 end

lisad avatar Nov 17 '13 19:11 lisad

Dynamodb batch queries are limited to 25 items, so the solution would also have to break large arrays up into 25 item batches.

lisad avatar Nov 21 '13 17:11 lisad

Hi Lisa,

That seems reasonable. I speculate the changes you need to make are twofold and about what you outline:

  1. Changes to criteria/chain.rb to properly detect that a batch_get is possible based on the where conditions
  2. Changes to adapter/aws_sdk.rb to be smart about using batch_get when given a set of keys

If you're up for putting together a PR, I suggest:

  • Write some specs first, try to cover both hash_key and hash & range cases.
  • Enumerable#each_slice makes it pretty trivial to group things into sets of 25.
  • See if there's any consolidation to be done with Dyamoid#find([id1, id2, …]). If you make chain.rb to just auto-detect when it can do batch gets, you can rewrite find() in terms of that and probably prune out some code elsewhere.

Hope this helps!

loganb avatar Nov 21 '13 19:11 loganb