Move semantic pointers

[mcmah309]

I just found out about this language and very happy that it exists. I've been toying with the idea of creating my own language but C3 seems to have a lot of ideas I thought of to improve apon C and others I have not. One idea I have that I would love to see is move semantic pointers. In which, code is run based on the move semantics of the pointer. This can enable RAII like idioms at the type level. By making ownership explicit in the type system, entire classes of bugs (use-after-free, double-free, memory leaks) can become compiler errors instead of runtime crashes. Unlike C++ smart pointers or Rust's borrow checker, this design stays optional and zero-cost—you only pay for what you use. It enables RAII-style resource management (files, locks, GPU buffers, etc.) while maintaining C3's philosophy of being transparent, predictable C.

Examples using C

---

E.g. a unique pointer (own/borrow) could be modeled like so:

void main(void) {
    {
        own Example* value = example();
    }
    // value goes out of scope here and clean up function pointer is called
}

// A annotation that tells the compiler this **can** be treated as an owned pointer.
// The first argument here is a function pointer to clean up code called when an `own` version goes out of scope.
[[owned(Example_cleanup)]]
typedef struct {} Example;

Example* Example_new() {
    return malloc(sizeof(Example));
}

void Example_cleanup(Example* value) {
    free(value);
}

own Example* example() {
    // Create a owned pointer like a regular pointer
    own Example* value = Example_new();
    // explicit borrow, valid since ownership is not past (takes borrow)
    use_example(borrow value);
    while(true) {
        // moves value, value in this function becomes a nulptr, but then it is instantly re-assigned so it is okay
        value = use_owned_example(move value);
        break;
    }
    return value;
}

// `borrow` is just a pointer annotation telling the function "You do not own what this is pointing to" and any using functions "This function will not take ownership of this value"
void use_example(borrow Example* value) {}
own Example* use_owned_example(own Example* value) {
    return value; // No clean up code called since the value is returned
}

---

E.g. a shared pointer (share) could be modeled like so:

void main(void) {
    {
        share RcInt* value = example();
    }
     // value goes out of scope here and clean up function pointer is called  (decrement refcount, refcount is now 0 so it is freed)
}

// A annotation that tells the compiler this **can** be treated as a shared pointer.
// The first argument here is a function pointer to call each time a value is shared - (`share value`).
// The second argument here is a function pointer to clean up code called for each `share` version that goes out of scope.
[[shared(RcInt_share, RcInt_destroy)]]
typedef struct {
    int count;
} RcInt;

share RcInt* RcInt_new() {
    RcInt* value = malloc(sizeof(RcInt));
    value->count = 1;
    return value;
}

// Called each time it is shared
void RcInt_share(share RcInt* value) {
    value->count++;
}

// Called each time it goes out of scope
void RcInt_destroy(share RcInt* value) {
    value->count--;
    if(value->count == 0) {
        free(value);
    }
}

share RcInt* example() {
    // Create a shared pointer like a regular pointer
    share RcInt* value = RcInt_new();
    // Explicit share - calls the share function pointer (increments refcount)
    use_example(share value);
    while(true) {
        // moves value, value in this function becomes a nulptr, but then it is instantly re-assigned so it is okay
        value = use_shared_example(move value);
        break;
    }
    return value;
}

void use_example(share RcInt* value) {
    // clean up code is called at end of scope (decrement refcount)
} 
share RcInt* use_shared_example(share RcInt* value) {
    return value; // No clean up code called since we return the value (no refcount change)
}

---

Using constructs like own/borrow and share are completely optional. These pointers are never coerced into a raw pointer (Only with an explicit cast, which is discouraged).

[joshring]

I am not an authority on these matters, but I note that RAII is included in the discussion in https://c3-lang.org/faq/rejected-ideas/

There are some alternative ideas in C3 to RAII like "trailing body macros" which are scopes where you can automatically manage cleanup for resources, like GPU buffers, mutexes, files etc

There are also things like the temporary allocator which can help managing memory, or arenas if you want more control. From what I have learned so far it looks like the emphasis is moving towards trying to create consolidated, organised and predictable lifetimes for memory and moving away from mallocs distributed across the heap. The traditional RAII is particularly well suited to these less well organised allocations but that type of allocation has some drawbacks in terms of memory locality, ease of understanding and performance.

[lerno]

@mcmah309 Thank you for this proposal. TLDR; I need to reject it, but I really like how they work as being opt-in for usage, so that some types are "prepped" for this usage. I haven't really seen quite that solution before, it's like opt-in destructors.

Let me point out some difficulties with C3. First of all this is dangerously close to regular destructors, and the sometimes unexpected ways they can trigger. However, the opt-in nature of it allows side-stepping it, so that is good.

However, let's say we have a struct Foo which contains an owned pointer, then struct Foo must also participate in ownership. I am uncertain how this is to be expressed at the syntactic level as it seems like the struct Foo is mandatory ownership tracked. This might mean that it might not be able to avoid opt in.

Secondly, programming patterns (which @joshring touched on) matters. In C++ back in the pre-smartpointer days, you would regularly see things like Foo* f = new Foo(a, b). Allocating objects on the heap was fairly common, and with that comes the problem of also making sure that those are freed properly.

If the code looks like this, then smart pointers is a boon. It allows retaining those allocation patterns with added safety and vastly less bookkeeping.

If you look at C3 code though, you'll see that returning a temporary allocation – which in C++ would be a std::unique_ptr or std::shared_ptr – uses the temp allocator, which doesn't require deallocation. So consequently usecases like: foo(bar(baz())) where baz() returns a pointer just works without actual ownership tracking.

I was experimenting with fat pointers that tracked ownership, and ultimately what made me reject them wasn't the destructor/constructor problems, but that they were trying to solve the same problem as the temp allocator. The difference is that the temp allocator doesn't require lots heap allocations, but are mostly just updating the temp allocator's underlying arena allocator.

What remained was the use case with long term ownership, and that was actually less useful with the fat pointers: if you have something you know runs for a long time, fixing that memory is often straightforward and easy to check for leaks.

In C++/Rust, ownership is a big thing because it can be used for both temp values and long term ownership. For C3 it would only be needed for the latter, so the cost of adding sufficiently good machinery for it seem too add too much complexity to the language. Or phrased differently: since C3 has already consumed its complexity budget, what features would you remove from C3 to put this in?

[mcmah309]

TLDR; I need to reject it,

I understand if that is the conclusion you come to. No worries, but happy to flesh out ideas here even so.

First of all this is dangerously close to regular destructors, and the sometimes unexpected ways they can trigger.

I also thought of this and acknowledge that being explicit everywhere is more of the "C way". The alternative to this, instead of automatically calling the cleanup code, it becomes a compile error where the user must specify to either forget or drop the value. e.g.

void main(void) {
    {
        own Example* value = example();
        // Compile error: `value` must be moved, or explicitly forgotten or dropped before going out of scope.
        // Solution: `drop value` or `forget value`
    }
}

Here the compiler is your friend, that reminds you you must explicitly handle this type. No unexpected triggers.

However, let's say we have a struct Foo which contains an owned pointer, then struct Foo must also participate in ownership. I am uncertain how this is to be expressed at the syntactic level as it seems like the struct Foo is mandatory ownership tracked.

Yes and no. One solution to allow non-mandatory tracked ownership is to treat passing an own pointer to a struct like passing it to a function - it becomes the structs duty to handle de-allocation. I envision the nesting scenario like so

// Container has an [[owned]] annotation AND contains an `own` field
[[owned(Container_cleanup)]]
typedef struct {
    own Example* data;  // Much like `borrow`, this is a hint, you should clean this up
} Container;

void Container_cleanup(Container* c) {
    Example_cleanup(c->data);
}

Note just because a struct contains and own pointer, does not mean it needs an owned annotation. This should be fine.

typedef struct {
    own Example* data; 
} Container;

One might even decide to cast the own pointer to a regular pointer, so the data structure looks like this

typedef struct {
    Example* data; 
} Container;

This allows own and share to be non-intrusive and might even play better with forget and drop as the compiler can help the user when it knows the value should be handled.

What this really means is own/borrow/share types become type hints when the compiler does not not know how to handle it, and enforce when it does - e.g. an own pointer goes out of scope. It should be recommended though that if a type contains an own pointer, one should also usually use that as own as well.

it looks like the emphasis is moving towards trying to create consolidated, organised and predictable lifetimes for memory and moving away from mallocs distributed across the heap.

If you look at C3 code though, you'll see that returning a temporary allocation – which in C++ would be a std::unique_ptr or std::shared_ptr – uses the temp allocator, which doesn't require deallocation. So consequently usecases like: foo(bar(baz())) where baz() returns a pointer just works without actual ownership tracking.

I see this as an alternative approach, not a conflicting approach. Much like how there are always more than one way to solve a problem. By all means, if the lifetimes are localized, then a temporary pool probably makes the most sense.

n C++/Rust, ownership is a big thing because it can be used for both temp values and long term ownership. For C3 it would only be needed for the latter

Yes, I do think this offers a good solution for long term and complex ownership. For localized ownership, pools probably make the most sense, but that doesn't exclude this approach either depending on the user and domain preferences. I see no reason, why a type could not be used like a regular pointer locally then converted (if in a pool) or casted to an own type if it needs to escape the locality.

since C3 has already consumed its complexity budget, what features would you remove from C3 to put this in?

I don't think anything needs to go to add this type of system. One could write performant valid C3 code and never use move semantic pointers. But for large projects where lifetimes become unwieldy. This could be a powerful non-intrusive approach for writing correct code.

[lerno]

If it's just a hint, it will offer much fewer guarantees and seemingly be rather useless? I had written something similar with "managed" pointers back as an old idea, mostly inspired by Cyclone. But in the end I just couldn't find enough use for it.

There is another thing I'd like to mention though. Working with std::shared_ptr in C++, I encountered a lot of problems in complex ownership code. This is something I also saw in ARC Objective-C code: in most cases, automatic refcounting just works, but then in some cases you want to defer or in other ways manipulate the refcount manually to achieve something, and that's where the pain starts when using automatic refcounting. ObjC has some tools like autorelease pools, but they're not useful across threads. It's not so much releasing object A which is the problem, but the fact that it holds a shared ref to B which holds a shared ref to C, and when C is released at that particular point in time it might be bad. With explicit refcounting, you can at least see these dependencies, but with automatic ref counting it's often difficult to see.

So even though this feature can be useful, it can also be an anti-feature.

In Rust, everything works in conjunction with these features, creating value that is greater than the sum of its parts. That would not be true in C3 from what I can see.

[mcmah309]

If it's just a hint, it will offer much fewer guarantees and seemingly be rather useless?

C also warns about a lot of problems that may or may not be errors. I wouldn’t consider that “useless” like returning the address of stack memory or possible dangling pointer. In the same way I imagine this would warn (or error depending on compiler flag) for situations it knows are errors.

It’s really nice for external programers too that they are aware of the ownership semantics of a pointer instead of trying to track down “do I need to free this or is this dangling at this point”.

With explicit refcounting, you can at least see these dependencies, but with automatic ref counting it's often difficult to see.

Everything in this proposal (if we include explicit drop) is explicit. So it is easy to see what is happening. Nothing is stopping someone from casting away the share or manually interacting with the ref count.

[lerno]

What I'm always careful of is adding more annotations or "viral" annotations (such as the infamous "const" in the case of C). Do you have some slim version of this with minimal annotation and only tracking (such that it errors if things are not handled, as opposed to implicitly doing things) that you would like to propose?