zig request: distinct types

would it be possible to add distinct types? for example, to make it an error to pass a GLuint representing a shader as the program for glAttachShader.

Sep 27 '18 06:09 emekoi

do you mean something like strong typedefs? https://arne-mertz.de/2016/11/stronger-types/

I'm strongly for this 😃 👍

Sep 27 '18 07:09 ghost

From the article posted by @monouser7dig:

They do not change the runtime code, but they can prevent a lot of errors at compile time.

Sounds like a job for comptime.

Sep 27 '18 11:09 nodefish

Certainly a good thing to have.

Sep 27 '18 11:09 XVilka

how would comptime provide this feature? what i mean is we could do something like const ShaderProgram = distinct u32; and it would be an compiler time error to pass a plain u32 as a ShaderProgram and vice versa.

Sep 27 '18 12:09 emekoi

The current workaround is (like c) to use a Struct with just one member and then always pass the Struct instead of the wrapped value. The big downside is that setting and getting the member is always boilerplate and does discourage the use of such a typesafe feature.

Sep 27 '18 13:09 ghost

Without yet commenting on the feature itself, if we were to do it, I would propose not changing any syntax, and instead adding a new builtin:

const ShaderProgram = @distinct(u32);

Sep 27 '18 13:09 andrewrk

how would comptime provide this feature?

You're right, I conflated this with the "strong typedefs" described in the article posted above. They are distinct concepts after all, no pun intended.

Sep 27 '18 14:09 nodefish

yeah, i think @distinct is a better than distinct.

Sep 27 '18 14:09 emekoi

I'm actually quite fond of this idea. Would there be any issues if we took it further and allowed functions to be declared inside?

// Pass a block like in @cImport()
const ShaderProgram = @distinct(u32, {
    pub fn bind() void { ... }
    pub fn unbind() void { ... }
});

or an alternative way with minimal changes to syntax that is consistent with enum semantics of 'underlying type'.

const ShaderProgram = struct(u32) {
    pub fn bind(sp: ShaderProgram) void { ... }
    pub fn unbind(sp: ShaderProgram) void { ... }
};

EDIT: Just a bit further - this could allow for explicit UFCS. The blocks below would be equivalent:

ShaderProgram(0).bind();
ShaderProgram(0).unbind();

var sp: = ShaderProgram(0);
sp.bind();
sp.unbind();

ShaderProgram.bind(0);
ShaderProgram.unbind(0);

Sep 28 '18 22:09 raulgrell

Nim has such feature and it is rather clumsy. Distinct type looses all associated operations (for example, distinct array type lacked even element access by []). All these operations have to be added, and there is lot of special syntax to "clone" them from original type. Nice idea was butchered by implementation.

Sep 28 '18 22:09 PavelVozenilek

@PavelVozenilek but thats because nim has operator overloading. in zig all the operators are known at compile time, so wouldn't the compiler be able to use the implementation of the distinct type's base type? for example:

const ShaderProgram = @distinct(u32); // produces a Distinct struct

const Distinct = struct {
    cont base_type = // ...
    value: base_type
};

and when an operator is invoked on the type the compiler can basically insert an @intCast(ShaderProgram.base_type, value.value) or the equivalent.

Sep 28 '18 22:09 emekoi

I think distinct types are useful, but I don't think they fit in Zig. There should be only one obvious way to do things if possible and reasonable.

The problem with distinct types is that you most likely don't want all of the operators or methods of the underlying type.

For example:

var first : = ShaderProgram(1);
var second : = ShaderProgram(2);
//This should be an error with all math operators
var nonsense : ShaderProgram = first *insert any operator here* second;

Sep 28 '18 22:09 Ilariel

void glAttachShader(GLuint program, GLuint shader);

I'm not familiar with the gl api, but I assume that program and shader are effectively opaque types. Despite being integers, it would not make sense to do any arithmetic on them, right? They're more like fd's in posix.

Perhaps we can scope this down to enable specifying the in-memory representation of an otherwise opaque type. There are two features that we want at the same time:

A library will provide and accepts objects of a type that the client isn't supposed to do anything else with. These objects function as handles.
The handle must have some concrete in-memory representation so that the client and library can communicate coherently.

The recommended way to do this is to make a type with @OpaqueType(), and then use single-item pointers to the type as the handle.

const Program = @OpaqueType();
const Shader = @OpaqueType();

pub fn glAttachShader(program: *Program, shader: *Shader) void {}

But this mandates that the in-memory representation of the handle is a pointer, which is equivalent to a usize. This is not always appropriate. Sometimes the handle type must be c_int instead, such as with posix fd's, and c_int and usize often have different size. You have to use the correct handle type, so a pointer to an opaque type is not appropriate with these handle types.

Proposal

A new builtin @OpaqueHandle(comptime T: type) type.

const H = @OpaqueHandle(T);
const G = @OpaqueHandle(T);

var t = somethingNormal();
var h = getH();
var h2 = getAnotherH();
var g = getG();

assert(H != T); - You get a different type than you passed in.
assert(G != H); - Similar to @OpaqueType(), each time you call it, you get a different type.
assert(@sizeOf(H) == @sizeOf(T) and @alignOf(H) == @alignOf(T)); - Same in-memory representation.
H is guaranteed to behave identically to T in the extern calling convention. This includes when it is part of a larger type, such as a field in an extern struct.
h = t; t = h; h = g; // all errors - The handle types don't implicitly cast to or from any other type.
if (h != h2) { h = h2; } - Handles can be copied and equality-compared.
h + 1, h + h2, h < h2 // all errors - Whether T supported arithmetic or not, the handle types do not support any kind of arithmetic.
t = @bitcast(T, h); - If you really need to get at the underlying representation, I think @bitcast() should be the way to do that. Or maybe we should add special builtins for this, idk.

This is an exciting idea. I think this fits nicely into the Zig philosophy of beating C at its own game - Zig is preferable to C even when interfacing with C libraries. If you translate your GL and Posix apis into Zig extern function declarations with opaque handle types, then interfacing with the api gets cleaner, clearer, less error prone, etc.

Sep 28 '18 23:09 thejoshwolfe

One objection I can think of to handling these as opaque types is that @distinct(T) as envisioned originally would be useful for C-style flags, and @OpaqueHandle(T) wouldn't because you can't use & and | with them without verbose casting.

Consider the following constants from win32 api

pub const WS_GROUP = c_long(131072);
pub const WS_HSCROLL = c_long(1048576);
pub const WS_ICONIC = WS_MINIMIZE;
pub const WS_MAXIMIZE = c_long(16777216);
pub const WS_MAXIMIZEBOX = c_long(65536);
pub const WS_MINIMIZE = c_long(536870912);
pub const WS_MINIMIZEBOX = c_long(131072);
pub const WS_OVERLAPPED = c_long(0);
pub const WS_OVERLAPPEDWINDOW = (WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX);
pub const WS_POPUP = c_long(-2147483648);
pub const WS_SIZEBOX = WS_THICKFRAME;
pub const WS_SYSMENU = c_long(524288);
pub const WS_TABSTOP = c_long(65536);
pub const WS_THICKFRAME = c_long(262144);
pub const WS_TILED = WS_OVERLAPPED;
pub const WS_VISIBLE = c_long(268435456);
pub const WS_VSCROLL = c_long(2097152);

and the following window creation code:

var winTest = CreateWindowExA(
    0,
    wcTest.lpszClassName,
    c"Zig Window Test",
    @intCast(c_ulong, WS_OVERLAPPED | WS_MINIMIZEBOX | WS_SYSMENU),
    CW_USEDEFAULT,
    CW_USEDEFAULT,
    800,
    600,
    null,
    null,
    hModule,
    null
) orelse exitErr("Couldn't create winTest");

It may be desirable ensure that APIs like this use a @distinct() type instead of a normal int constant, to ensure that you do not accidentally pass something like WS_S_ASYNC, which is completely unrelated, or a (perhaps mis-spelled) variable containing an unrelated integer.

With @OpaqueHandle(T), the user could not use the function properly without casting the handles in a very verbose manner. This could be abstracted away by the API, though, by providing a varargs fn that would do that for you. Just something to consider since this is the usecase that sprang immediately to mind when I read the original proposal.

Sep 29 '18 04:09 tgschultz

I think these are two very different use cases and they might not have the same solution. I do like the @opaqueHandle it's useful as well outside the C interfacing use case, to create handles to other things. I have exactly this in my C++ project where the client gets handles to objects it creates (which is just an integer in a struct).

The flags use case might be solved with a special flag type, just like enums but supports bitwise operators or something.

Op za 29 sep. 2018 06:44 schreef tgschultz [email protected]:

One objection I can think of to handling these as opaque types is that @distinct(T) as envisioned originally would be useful for C-style flags, and @OpaqueHandle(T) wouldn't because you can't use & and | with them without verbose casting.

Consider the following constants from win32 api

pub const WS_GROUP = c_long(131072); pub const WS_HSCROLL = c_long(1048576); pub const WS_ICONIC = WS_MINIMIZE; pub const WS_MAXIMIZE = c_long(16777216); pub const WS_MAXIMIZEBOX = c_long(65536); pub const WS_MINIMIZE = c_long(536870912); pub const WS_MINIMIZEBOX = c_long(131072); pub const WS_OVERLAPPED = c_long(0); pub const WS_OVERLAPPEDWINDOW = (WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX); pub const WS_POPUP = c_long(-2147483648); pub const WS_SIZEBOX = WS_THICKFRAME; pub const WS_SYSMENU = c_long(524288); pub const WS_TABSTOP = c_long(65536); pub const WS_THICKFRAME = c_long(262144); pub const WS_TILED = WS_OVERLAPPED; pub const WS_VISIBLE = c_long(268435456); pub const WS_VSCROLL = c_long(2097152);

and the following window creation code:

var winTest = CreateWindowExA( 0, wcTest.lpszClassName, c"Zig Window Test", @intCast(c_ulong, WS_OVERLAPPED | WS_MINIMIZEBOX | WS_SYSMENU), CW_USEDEFAULT, CW_USEDEFAULT, 800, 600, null, null, hModule, null ) orelse exitErr("Couldn't create winTest");

It may be desirable ensure that APIs like this use a @distinct https://github.com/distinct() type instead of a normal int constant, to ensure that you do not accidentally pass something like WS_S_ASYNC, which is completely unrelated, or a (perhaps mis-spelled) variable containing an unrelated integer.

With @OpaqueHandle(T), the user could not use the function properly without casting the handles in a very verbose manner. This could be abstracted away by the API, though, by providing a varargs fn that would do that for you. Just something to consider since this is the usecase that sprang immediately to mind when I read the original proposal.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ziglang/zig/issues/1595#issuecomment-425615618, or mute the thread https://github.com/notifications/unsubscribe-auth/AL-sGuZxTFMx5aawVhueT-oWDPD9kwDpks5ufvrKgaJpZM4W79q6 .

Sep 29 '18 06:09 BarabasGitHub

Another use case: const Vec2 = [2]f32;

Sep 29 '18 08:09 raulgrell

In go, the new type inherits the operations but not the methods of the old type, I think this is a good way to do it as it provides the benefit without great complexity, type system does not need to catch every possible error, but just help us. https://golang.org/ref/spec#Type_declarations

Sep 29 '18 09:09 ghost

I agree that bitflags are a separate issue. I've partially typed up a proposal for bitflags in Zig including extern bitflags appropriate for interfacing with C. Those WS_GROUP etc constants as well as http://wiki.libsdl.org/SDL_WindowFlags could be represented in Zig as this new bitflags type, and that would also lead to beating C at its own game. The proposal ended up being pretty complicated, so I haven't posted it anywhere yet.

I think the usecase for a handle type is still valid separate from the flags case.

Sep 30 '18 15:09 thejoshwolfe

Wouldn't it be great if you can say that a function can receive either type A or B in a type safe way?

pub fn foo(myparam : u32 or []const u8) {
}

I know what the critique against it is: "just make a parent struct". But that's not the point, this gets the job done so much faster without all the boilerplate of constantly writing structs and naming them and setting them up even though I don't actually need a struct. I'm usually always against adding any kind of type shenanigans but this is actually something I use and need all the time in languages like Typescript.

Oct 18 '18 06:10 Meai1

@Meai1 I'm not sure how this solves the issue. We're talking about allowing two names A and B, to have the same underlying type (usize or something else) but disallow implicit casts between them.

I think what you're proposing fits with #1268.

Oct 18 '18 06:10 Hejsil

@Hejsil Because I think that what is described in this issue is just a tiny subset of the general problem/solution of "type refinement": https://flow.org/en/docs/lang/refinements/

edit: I guess they call it 'subtyping' when it is used to define something, in my opinion they look identical: https://flow.org/en/docs/lang/subtypes/

Oct 18 '18 07:10 Meai1

what the first article talks about are sum types which can be achieved through union types. as for subtypes i don't see how that relates to distinct types. what i meant by distinct types is that when B is a distinct A, that are the same type but A cannot implicitly cast to B and vice versa. this means calling a function with the signature fn foo(bar: A) void with an argument that is of type B is an error, despite the fact types A and B are identical.

Oct 18 '18 16:10 emekoi

I think @thejoshwolfe's proposal is promising. One modification though:

t = @bitcast(T, h); - If you really need to get at the underlying representation, I think @bitcast() should be the way to do that. Or maybe we should add special builtins for this, idk.

Following with the established pattern, opaque handles would have their own casting functions, unique to opaque handles. @fromOpaqueHandle(x) to get the value, @toOpaqueHandle(T, x) to get the opaque handle.

The strongest argument against this I can think of is that it is More Language :-1: . The counter-argument is that it Clearly Prevents Bugs :+1:

Here's a question to consider: in a pure zig codebase, would there be a reason to use @OpaqueHandle?

Oct 18 '18 18:10 andrewrk

Whether T supported arithmetic or not, the handle types do not support any kind of arithmetic.

I think this is a bad idea, because its very verbose so people will not use it (enough) https://github.com/ziglang/zig/issues/1595#issuecomment-425632219

Here's a question to consider: in a pure zig codebase, would there be a reason to use @OpaqueHandle?

everywhere you use an int/ float type

Oct 18 '18 20:10 ghost

If we're not gonna support the operators for the type, then we are pretty close to be able to have this in userland:

const std = @import("std");
const debug = std.debug;

pub fn OpaqueHandle(comptime T: type, comptime hack_around_comptime_cache: comptime_int) type {
    return packed struct.{
        // We could store this variable as a @IntType(false, @sizeOf(T) * 8)
        // but we lose the exact size in bits this way. If we had @sizeOfBits,
        // this would work better.
        ____________: T,

        pub fn init(v: T) @This() {
            return @This().{.____________ = v};
        }

        pub fn cast(self: @This()) T {
            return self.____________;
        }
    };
}


test "OpaqueHandle" {
    // I know that the 0,1 things is not ideal, but really, you're not gonna have
    // 10 or more of these types, so it's probably fine.
    const A = OpaqueHandle(u64, 0);
    const B = OpaqueHandle(u64, 1);
    debug.assert(A != B);
    const a = A.init(10);
    const b = B.init(10);
    debug.assert(a.cast() == b.cast());
}

Here's a question to consider: in a pure zig codebase, would there be a reason to use @OpaqueHandle?

I'm pretty sure I'd never use this.

Nov 02 '18 10:11 Hejsil

comptime hack_around_comptime_cache: comptime_int this could be a type and then you pass @OpaqueType() rather than 0, 1, etc.

Nov 02 '18 14:11 andrewrk

@andrewrk Aaah, nice!

Nov 02 '18 14:11 Hejsil

A distinct integer type seem like it could be done via an empty non-exhaustive enum (#2524).

The example from earlier using a non-exhaustive enum:

const ShaderProgram = enum(u32) {
    _,
    pub fn bind(sp: ShaderProgram) void { ... }
    pub fn unbind(sp: ShaderProgram) void { ... }
};

You could get to the underlying integer via @enumToInt and @intToEnum. The enum doesn't have to be empty either, it would let you specify "special" values if your distinct integer type has them.

May 23 '19 02:05 daurnimator

how is this bloat? having distinct types goes along one of the zens of zig: "communicate intent precisely". this is a proposal for the type system to help catch errors at compile-time.

Jun 20 '19 21:06 emekoi

Here's one of the usecases in my emulator.

[]u8:

Registers
Disassembly
Data in memory
Pixels

[]u16:

Registers
Memory addresses

Using raw values, it was a struggle to correctly distinguish registers and memory data, until I wrapped everything in a struct. Granted this was C so even mixing up pointers and arrays was a nightmare.

Being able to add methods in Zig was actually quite powerful so I'd love to see this concept embraced instead of worked around.

Jul 09 '19 01:07 fengb

zig zig copied to clipboard

request: distinct types

Proposal

zig
zig copied to clipboard