crossbeam icon indicating copy to clipboard operation
crossbeam copied to clipboard

Make crossbeam::deque::Worker::push return index of relative slot written

Open Bajix opened this issue 3 years ago • 3 comments

It is useful to know when crossbeam::deque::Worker::push adds the first task to an empty deque as this can be used to facilitate simple batching by triggering tokio::task::spawn and work stealing to completion.

Here's an example of the how this change could be used

pub fn batch_update_cache<K, V>(runtime_handle: &Handle, data: (String, Vec<u8>)) {
  thread_local! {
    static QUEUE: Worker<(String, Vec<u8>)> = Worker::new_fifo();
  }

  QUEUE.with(|queue| {
    if queue.push(data).eq(&0) {
      let stealer = queue.stealer();

      runtime_handle.spawn(async move {
        let mut cache = vec![];

        cache.extend(std::iter::from_fn(|| loop {
          match stealer.steal() {
            Steal::Success(data) => break Some(data),
            Steal::Retry => continue,
            Steal::Empty => break None,
          }
        }));

        /// do something with cache
      });
    }
  });
}

Bajix avatar Sep 04 '21 20:09 Bajix

How do we feel about this change? I could update the PR to make the same change for other data structures as well

Bajix avatar Sep 16 '21 00:09 Bajix

  • First, this is a breaking change.

Actually it's not a breaking change; we can assume that as this currently returns a unit type, everywhere that currently use this does so without assignment, and as there's no addition of a #[must_use] annotation, this usage will continue to work without change.

  • Second, if I understand correctly, the existing features (len, is_empty, etc.) seem to be sufficient for your use-case here.

Those operations aren't useful for my needs. I specifically need to be able to know if an element was pushed while empty, but if I just do a push and then check len immediately thereafter, there's the possibility that another thread since pushed, and should that happen, there's no way to determine which thread was actually first to push. This PR ensures that my logic isn't prone to race conditions.

Bajix avatar Sep 19 '21 05:09 Bajix

Actually it's not a breaking change; we can assume that as this currently returns a unit type, everywhere that currently use this does so without assignment, and as there's no addition of a #[must_use] annotation, this usage will continue to work without change.

No, this is breaking because an expression that returns unit (()) can omit the semicolon.

Those operations aren't useful for my needs. I specifically need to be able to know if an element was pushed while empty, but if I just do a push and then check len immediately thereafter, there's the possibility that another thread since pushed, and should that happen, there's no way to determine which thread was actually first to push. This PR ensures that my logic isn't prone to race conditions.

That makes sense.

taiki-e avatar Nov 03 '21 16:11 taiki-e