heapless icon indicating copy to clipboard operation
heapless copied to clipboard

Would `heapless` accept a PR for a `Deque`/`RingBuffer` that operates on a mutable slice instead of allocating a buffer of a fixed size?

Open ColinFinck opened this issue 4 months ago • 4 comments

I basically need smoltcp's RingBuffer in a dedicated library, because I want to use it in a non-networking context. You currently cannot use it without adding a dependency to smoltcp and its network features.

One idea is to move it from smoltcp to a dedicated crate, and I have sent out an inquiry whether they would accept a PR in that direction. Another idea is to use a similar data structure from another crate. I have looked at the Deque of heapless, but it requires a capacity type parameter. smoltcp's RingBuffer asks the user to provide a mutable slice. The same RingBuffer can therefore operate on buffers of arbitrary size. This is exactly what I need for my application.

Now I would be happy with any data structure that may be used like a ring buffer, as long as it operates on mutable slices. Be it another Deque-like struct that shares much of its API (what would be a good name for that?), or taking over the original "RingBuffer" with its API.

Before I come up with a PR: Which of these options would the heapless crate accept?

ColinFinck avatar Aug 15 '25 08:08 ColinFinck

I have sent out an inquiry whether they would accept a PR in that direction.

Could you please point out to where you asked? I don't see anything here. 🤔

have looked at the Deque of heapless, but it requires a capacity type parameter.

All of the heapless API is geared toward fixed-sized buffers on the stack. So even if we were to add an API to allow user to provide a mutable slice, we would still need to support the existing behaviour/API and therefore still have the const generic on types.

zeenix avatar Aug 17 '25 16:08 zeenix

The existing View API could be updated to support that use case somewhat. Maybe we can add a function gated by the alloc feature with an API akin to fn boxed_view(capacity: LenT) -> Box<DequeView<T, LenT>> for all the various types View variants. Would this support your use case?

We could also adapt the Storage traits so that they accept &mut [MaybeUninit<T>] as the storage but this type of storage wouldn't be convertible to a View, so the storage trait would need two variants (one that can be coerced into a view, and one that can't).

sgued avatar Aug 18 '25 07:08 sgued

@zeenix:

Could you please point out to where you asked? I don't see anything here. 🤔

You're right. I only asked a maintainer by e-mail, but without any luck so far. Opened an issue now: https://github.com/smoltcp-rs/smoltcp/issues/1082

@sgued:

The existing View API could be updated to support that use case somewhat. Maybe we can add a function gated by the alloc feature with an API akin to fn boxed_view(capacity: LenT) -> Box<DequeView<T, LenT>> for all the various types View variants. Would this support your use case?

My use-case is still a statically allocated buffer from the stack. Let me give a code example:

struct MyObject<'a> {
    queue: RingBuffer<'a, u32>
}

let mut buffer1 = [Default::default(); 10];
let object1 = MyObject { queue: RingBuffer::new(&mut buffer1[..] };

let mut buffer2 = [Default::default(); 20];
let object2 = MyObject { queue: RingBuffer::new(&mut buffer2[..] };

let all_objects = [object1, object2];

You see, my buffers are still allocated on the stack. But if I used Deque here, the static size would be part of the Deque type. all_objects could not contain object1 and object2 then. This is also a problem for many other scenarios (e.g. passing a buffer to a dynamically dispatched API).

ColinFinck avatar Aug 18 '25 13:08 ColinFinck

The View types that should be released very soon as part of 0.9 will solve that. You will still need the const generic when allocating it on the stack but then you can pass around a reference to the deque type without having to worry about the const generic.

This compiles on latest main:

        trait SomeTrait {
            fn works_with_deque(&mut self, deque: &mut DequeView<u32>);
        }

        let mut deque = Deque::<u32, 10>::new();
        let mut dyn_obj: &mut dyn SomeTrait = todo!();

        // Here the deque gets coerced into a `&mut DequeView<u32>` that doesn't have a const generic.
        dyn_obj.works_with_deque(&mut deque);

sgued avatar Aug 18 '25 13:08 sgued