William Wen
William Wen
> The recovery mechanism is being refactored. See https://github.com/risingwavelabs/risingwave/pull/5396#issuecomment-1249395039 Do we need to wait for that before working on this issue? I think we can start working on the storage...
Neither ReadVersion nor uploader handles the buffer release. They are just holding a reference to the buffer. The buffer release is handled by reference counting, which is, only when no...
> One issue is start_buffer_tracker_worker checks Arc::strong_count(&local_version_manager) > 1, while start_pin_worker/start_unpin_worker convert weak ref to strong one on demand. They may prevent each other from exiting. Since the `start_buffer_tracker_worker` is...
> start_buffer_tracker_worker exits when local version manager's ref_count is 1, which is almost impossible to be met because each HummockStorage holds a ref to local_version_manager, and HummockStorage is cloned many...
https://github.com/risingwavelabs/risingwave/pull/5469 will now shutdown the `buffer_tracker_worker` when there is no externally usage on `LocalVersionManager`. In our current code, a potential unstoppable external usage on `LocalVersionManager` will be the notification observer....
> By the way, if there's no dyn Future on the path, I think the compiler is able to realize that and the generated state machine should be the same...
> I would prefer not naming it as poll_next, and don't use std::task::Poll here to avoid confusion. Do you have any suggestion on the naming? I have been thinking about...
> How about use poll_next in compact ? For example: ``` async fn compact(...) { while iter.is_valid() { iter.poll_next(); if iter.is_pending() { iter.await_next().await; } } } ``` ****** The `next`...
> Yes, so I'm suggesting to use multipart upload API. Sure I think we can do multipart upload. I had some rough idea before about providing a new upload API...
> I think from our perspective, object store interface implemented for S3 should pull bytes from input stream. If input stream not exhausted, add to buffer of fixed size (min...