API to replace duk_put_function_list() and duk_put_number_list()

[svaarala]

There are two API calls to conveniently initialize objects from C:

• http://duktape.org/api.html#duk_put_function_list
• http://duktape.org/api.html#duk_put_number_list

There are a few downsides to these:

• Separate calls are needed to initialize functions and numbers
• There's no support for all common types (e.g. booleans)
• There's no support for lightfunc initializers

Also, the current approach exposes two specific C structs used as the initializer, which was probably a mistake: C structures are difficult to extend in a binary compatible fashion. A more extensible approach would be to use macros for both declaring the initializer array and for initializer entries; this would allow the underlying structures to be extended to e.g. support lightfuncs. Also the same API call could provide initializers for both functions and numbers (and other constants).

Tasks:

• [x] Figure out the best initializer macro model
• [ ] Minimum property types: undefined, null, boolean, number, string, function, lightfunc
• [ ] Nice-to-have property types: buffer, pointer, function-with-magic #574
• [ ] Property attribute support; rename to duk_def_prop_list()?

[svaarala]

One problem with using initializer macros is that the most natural initializer would involve a type/flags field and a union of some sort for the value alternatives. But union initializers are not portable to pre-C99 compilers.

[svaarala]

Having the ability to provide property attributes too (if user wants to use non-default values) would also be useful.

[svaarala]

One option for portable initialization is to:

• Use unions as the property list value type
• Use named union initializers in C99 compilers, as they're quite widely supported
• Fall back to using helper functions to fill out the entries for other compilers

Example of a helper:

duk_proplist_union duk_make_proplist_number(const char *key, duk_double_t numval) {
    duk_proplist_union u;
    u.key = key;
    u.number = numval;
    return u;
}

which would then be used in the initializer like:

    ...
    /* expanded from DUK_PROP_NUMBER */
    duk_make_proplist_helper("myprop", 123.0),
    ...

C99 compilers would just use the initializer:

    ...
    /* expanded from DUK_PROP_NUMBER */
    { .key = (key), .number = (numval) },
    ...

For both the initializer list entry would look identical, e.g.:

    ...
    DUK_PROP_NUMBER("myprop", 123.0),
    ...

[svaarala]

Another portable option which is a lot messier is to use a struct initializer, but share the fields so that e.g. both void and string pointers can share the same field in the struct. This doesn't result in as optimal a layout as the union approach because there would be both data and function pointers in the struct etc.

[svaarala]

Here's a very basic sketch of how that union initializer/helper call model would work:

• https://github.com/svaarala/duktape/compare/api-new-list-inits

This is just to document what the most potential approach seems to be so far; I'll return to this later on for Duktape 1.4.

[svaarala]

Helper functions may be an issue for global values. Perhaps the best approach is to:

• Use union initializers for C99
• Use struct initializers for non-C99 environments, with some struct members reused where possible (e.g. plain pointer and string pointer can share a field; function pointers can't) but with a little bit larger struct than the union

This would be quite portable, has static initialization (avoids helper function calls).

[darkuranium]

There is a problem with the helper functions w.r.t. global variables. On the other hand, { .field = value } syntax is not available in all compilers (despite it being standard: C99), so it cannot all be done with unions+macros.

MSVC is one notable compiler without explicit union field initializers (the recent versions may have fixed that by now --- can someone who has the compiler confirm?).

What I propose is using unions where possible, and structs where not. Note that this makes Duktape compiled with MSVC will no longer be ABI-compatible with that compiled with GCC or clang (not without a proper HAVE_* override, at any rate) --- assuming it was compatible to begin with.

Of course, an alternative, cleaner way is to just use structs through and through; unfortunately, the struct approach uses quite a lot more space compared to unions (almost twice as much in amd64: 72 vs 40 bytes per item).

Here is my proposal/prototype API: https://gist.github.com/darkuranium/565a8526abd2de078371 (I've used the xduk_ prefix to prevent my own API from clashing with the official Duktape API, assuming someone wants to copy&implement this for themselves). There is no implementation for xduk_put_prop_list() written at the moment, but it's just a bunch of duk_def_prop() calls in a loop.

[svaarala]

@darkuranium I agree that union + struct initializer fallback seems like the best option unless someone comes up with a better idea.

[fatcerberus]

I have MSVC 2015, I should check that out.

[darkuranium]

@fatcerberus: That's a start, though I think it's important to know if/which older versions are supported, too. There's also the matter of compilers for embedded devices, some of those are archaic and might not support C99.

[svaarala]

I'll try to get this issue resolved for Duktape 1.5.0, so far union+struct approach (see example by @darkuranium) seems to be the most realistic option. I'll try to come up with a pull which adds config options etc to the approach.

[darkuranium]

@fatcerberus: Any updates w.r.t. MSVC 2015 support for C99? Specifically, the support for named field initializers (e.g. { .foo = 5, .bar = 7 } for struct { int foo, bar; })?

---

@svaarala: A reminder (just in case): keep in mind that DUK_PROP_*() macros must remain macros, and not functions (even though this would simplify matters a lot!). This is because they must be usable in constant initializers, e.g. [https://gist.github.com/darkuranium/565a8526abd2de078371#file-duk_def_prop_list-h-L100-L111(lines 100-111 in the proposal I've pasted above).

You could save some space by combining some numeric fields, but at some cost, especially in platforms without a FPU (due to a float->int conversion) --- see the comments in my proposal, lines 52 and 56. Up to you, I'm just tossing some ideas out there.

[fatcerberus]

@darkuranium Sorry, I forgot all about this issue. :sweat_smile: I just checked, and the following compiles and runs successfully using MSVC 2015/cl.exe v19.00:

#include <stdio.h>

struct eaty { int pig, cow; };

int main()
{
    struct eaty eaty = { .pig = 812, .cow = 1208 };
    printf("pig: %d, cow: %d\n", eaty.pig, eaty.cow);
}

For some reason MSVC's prettifier wants to delete the space between the comma and dot, but named fields in initializers do indeed work.

[svaarala]

@darkuranium Re: macros vs. functions, I'm using union/struct initializer macros only. The helper functions are not going to be used because they don't work well for static initializers.

[svaarala]

@fatcerberus Any idea what's the first MSVC that supports the named union initializers?

[fatcerberus]

@svaarala I don't have a copy of MSVC 2013 installed anywhere, but I did some quick Google searching and it looks like MSVC 2013 was the first version to support them.

[svaarala]

@fatcerberus Thanks, I'll make DUK_USE_UNION_INITIALIZERS true for MSVC >= 2013 (the change will be in #575).

[svaarala]

Hmm: https://connect.microsoft.com/VisualStudio/feedback/details/805981:

(MSVC 2013) C99 Designated Initializers cannot initialize unions within structs

This would be an issue because the property initializers involve a union inside a struct.

I guess it's fixed in a newer MSVC 2013 version so I'll make the check for VS2013 for now.

[svaarala]

Actually it seems that the above bug was fixed in VS2015:

https://blogs.msdn.microsoft.com/vcblog/2015/07/01/c-compiler-front-end-fixes-in-vs2015/

So I'll make the check VS2015 unless someone can confirm it's also fixed in VS2013 (and how to check that the fix is in place).

[svaarala]

I've moved this to 2.1.0 because I haven't had time to complete the latest representation prototype. This means that duk_put_number_list() and duk_put_function_list() will be kept in 2.1.0 and removed in 3.0.0, which is fine, as they're not part of the binary of a footprint not actually calling them (Duktape doesn't call them internally).