Dynamoid
Dynamoid copied to clipboard
Support for ActiveRecord "subset conditions"
ActiveRecord supports "subset conditions", e.g. http://edgeguides.rubyonrails.org/active_record_querying.html has the example:
Client.where(orders_count: [1,3,5])
DynamoDB supports "BatchGet" : http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/DynamoDB/BatchGet.html
It would seem to be a good matchup. Dynamoid does not currently support this. When I attempt the DynamoidThing.where(id: array_of_ids) syntax, it fails in Dynamoid::Adapter::AwsSdk.query, which is documented to only support one hash key.
If this is a good fit and Dynamoid should be extended to support it, any suggestions how to integrate it? I would propose either in Dynamoid::Adapter.query or Dynamoid::Adapter::AwsSdk.query, detect the Array passed in and branch off the logic. Then in Dynamoid::Adapter::AwsSdk, create a new "batch_query" method that uses batch_get something like this (untested, just a sketch)
def batch_query(table_name, hash_keys)
items = hash_keys.map { |key| Item.new(:hash_value => key) }
batch_get.table(table_name, :all, items)
end
Dynamodb batch queries are limited to 25 items, so the solution would also have to break large arrays up into 25 item batches.
Hi Lisa,
That seems reasonable. I speculate the changes you need to make are twofold and about what you outline:
- Changes to criteria/chain.rb to properly detect that a batch_get is possible based on the where conditions
- Changes to adapter/aws_sdk.rb to be smart about using batch_get when given a set of keys
If you're up for putting together a PR, I suggest:
- Write some specs first, try to cover both hash_key and hash & range cases.
- Enumerable#each_slice makes it pretty trivial to group things into sets of 25.
- See if there's any consolidation to be done with Dyamoid#find([id1, id2, …]). If you make chain.rb to just auto-detect when it can do batch gets, you can rewrite find() in terms of that and probably prune out some code elsewhere.
Hope this helps!