hbase-stargate
hbase-stargate copied to clipboard
scanner splits rows depending on batch size
i have code similiar to the following:
scanner = client.open_scanner("table", {:columns => "a", :batch => 10, :limit => 4})
rows = client.get_rows(scanner, 10)
rows.each do |row|
puts row.name + " " + row.columns.to_s
end
when column-family 'a' has 4 columns this should return results like:
row1 col a:1 col a:2 col a:3 col a:4
row2 col a:1 col a:2 col a:3 col a:4
row3 col a:1 col a:2 col a:3 col a:4
row4 col a:1 col a:2 col a:3 col a:4
but what i get is :
row1 col a:1 col a:2 col a:3 col a:4
row2 col a:1 col a:2 col a:3 col a:4
row3 col a:1 col a:2
row3 col a:3 col a:4
row4 col a:1 col a:2 col a:3 col a:4
however, when I change :batch from 10 to 100 it works. It seams that rows are not correctly assembled when they are split because of the batch-size
I've also encountered this. This seems to originate in the behavior of HBase REST API itself, so it's not directly fault of this hruby-stargate. Still it doesn't mean that hruby-stargate shouldn't provide this higher-level convenience feature of re-uniting the rows, but, I wonder if REST API provides the lower-level facilities for that (e.g. does it mention that particular rows incomplete and would be completed by the next iteration)