client-rust
client-rust copied to clipboard
Add new stream scan APIs for transaction
Add new scan_stream, scan_keys_stream, scan_reverse_stream, scan_keys_reverse_stream APIs for transaction scan.
Use these APIs, scan request will be invoke request to one region at a time in a configurable batch size, which is more reasonable when user scan range is wide-range, such as all key space.
@ekexium @iosmanthus PTAL
IIRC, you can limit the scan size in batch using the current API like:
let mut start_key = Key::from(vec![]);
let end_key = Key::from(vec![]);
let limit = 100;
let mut buffer = vec![];
let mut last_len = 0;
loop {
buffer.extend(client.scan((start_key, end_key), limit).await?);
if buffer.len() - last_len < limit {
break;
}
last_len = buffer.len();
start_key = buffer.last().unwrap().0.clone();
start_key.push_zero();
}
I agree with you that an iterative api is more ergonomic, but I think we can make it a wrap function like above instead of internal mechinism like in this PR.