Taking a look at how to effectively further the project

[ozra]

Since time hasn't been on my side lately, and for some more months likely, I found the reason to take a hard look at the project as realistically as possible to see how it should best be furthered.

tl;dr: - search for "The Obvious Candidate" header further down.

Will quickly discuss some linguistic goals in consort with practical goals.

My primary original motivations in creating a new language were of:

• Necessity - all available languages suck in one way or another in my eyes (and Onyx probably will to some extent too - just a lot less!) so naturally there was only one thing to do. (This is how mankind got out of the caves — so don't even try pulling the NIH card on me!)
• Primarily: Be fully usable replacement for C++ - which means those are the minimum feature powers acceptable.
  • Compilation time of clang/gcc was a secondary motivation.
• Allowing absolutely minimal amounts of formalism to get a program running
• Have as many constructs as needed to formalize code when wanted: making it either
  • Safer with stronger promises
  • Cycle-squeezing-fast and memory-fiddling-and-touching-minimized and tightened (the latter being the most important in today's common CPU's)
  • Able to do all the stuff with machinery that machinery can do, that higher level languages just ignores.
  • Or all of the above

Now, I need to take a hard look and re-focus how to realistically reach practical goals as of now. A plan has landed.

A primary practical take on Onyx dev...

• ...is to share a library universe with another language
  • I want to make a better language - not spend time rewriting basic definitions of types and functions that has been written again and again and...
• In Onyx (though this is just a continuation of a project that has gone under many names over a total span of almost 20 years), the Crystal lib universe was the candidate because Crystal itself seemed as a good enough (best practical candidate that could be found) to jump start the humty-eleventh implementation of the pet.
  • "Onyx" was the "branding" chosen in respect of "Crystal". Also the "X" makes it sound cool. The name will remain with the project indefinitely
  • Unfortuntely I really dislike many choices in the Ruby-API-based lib! [A "will just have to live with it" situation])
  • I was overly optimistic that the project would be open to good (out of my use-case and needs) suggestions.
    • Crystal's a very impressive project! Unfortunately not at all folding out in the directions that Onyx desires and requires — my bad!
  • The lib-sharing ideal made it necessary to fully handle understanding Crystal code.
    • The Onyx parts possible in concort (without making maintainance impossibly time consuming) were written - juggling Onyx code and Crystal together in the same program.
      • Still, a lot of effort goes in to maintaining cross-lang compatibility
      • A lot of favoured semantics has had to be eliminated to not make co-existance painful (projected scenario of spending 90% of time invested in maintaining inter-op rather than making good shit)
  • The differences in goals are becoming too obvious to ignore any longer.
    • It simply ain't a marriage made in heaven.
    • I won't list details here, to keep this post from resulting in tooooooooo l;dr:

So, naturally - to stick with a rational and realistic stiff upper lip attitude — it's obvious that prospecting a new "companion language universe" is imperative in order to reach intended goals with sane amounts of time invested. One that goes better together with Onyx; or the other way around more specifically.

Seeing as the origins of the project was a C-based parser ca '97, which then stalled, and then in the dark months of '99, out of desperation of being freed from forced braces-syntax — I hacked together Cython (not to be confused with another proj a few years later with the same name...). With emphasis on "hacked". It was an just-get-it-done heuristic line-by-line ordeal in Perl.

And of course that just had to become part of my everyday code bases. There's a lesson (of cooooourse):

"Get shit done" gets more shit done than, err... not getting shit done!

You can quote me on that one! (Just don't use my name)

The code did suck, but it actually plain just worked! Completely transparently in my everyday coding! So...

Obviously this is a time to take a step back. Or 18 years. Well, ok, not really. The point is: C++ should be the "family" (it's not it, but its' syntax I've loathed). And two parallel development paths should be taken.

The Obvious Candidate

• While losing ground during the first Y2K decade+ - C++ has made a rapid comeback in power with C++11, C++14 and now C++17.
• For me it's always been the one language I always return to - no matter how disgusting the syntax.
• It used to be a necessary evil, because it was the most reasonable way of getting shit performent.
• With C++11 it finally became a "pretty fucking capable beast"
• The new Iso C++ std-definitions every three years are going in a very good direction, and keeps the pace without delays so far.
• Compiler vendors are keeping the pace really well too! Which means that the tandem will likely continue solidly.

The arrived plan landed on a two development paths approach

Primary path: "Must be useable now!": Onyx-CX

• Is a thin frontend delegating all deeper semantics entirely to C++-backend
• In practise: leaves 95% of the Onyx compiler unused for this "edition" (it comes to play in the other)
• Type inference will be on par or likely better than current Crystal-friendly implementation - albeit lacking the "full-strength" type system features of Onyx.
  • reduced to C++ not-quite-as-strong-type-system (nulls can fuck you over if you work a lot with raw pointers [don't! There's really not much need] - it shouldn't be as bad as Java, C# or Go at least).
• Works solely using C++11/14/17+ (selectively based on chosen back-chain status) as highlevel IR
• Ensures any system can be targeted today — not two years from now; including 8-bit systems. Avr, Arm, X86*, static builds with musl for completely self contained executables, etc
• use any C++ and C-libs with zero effort
• Much more powerful meta coding is made possible than currently is!
• A lot of adaptations of syntax made in favour of Crystal compatibility will be removed, some additional will be introduced to make use of all power possible to harness.
• A bunch of "ghost-syntax" in Onyx can now be enabled as real syntax, since without Cr-compat in mind, it now becomes possible to do.
• Billions (literally!) of concurrent coroutines is made possible instead of just hundreds of thousands
• Threading is immediately possible, with several ways of doing it, not only CSP.
• Statically linked self-contained musl based binaries are made possible!

Secondary path: Long term goal. Onyx-XL (yes! I actually think "X" sounds cool!)

• Three (four [five]) backends (build-binary, render-C++ [=> render-asmjs], render-Lua, render-ES)
• A very small base Onyx stdlib will be available (intended to stay small, and no need to use it it!) only abstracting the most commonly used fundamental types and functions. Allows writing some shareable code for all backends.
  • The general rule is "the backend target is the platform"
• The primary and most important one: true superset of C++
  • This will allow the Onyx'y strong type features united with the C++ and C semantics (much like is done now with Crystal code being seamlessly used in the same project)
  • use C++ and C-libs with zero effort
  • enjoy true strong type safety and the type features of Onyx
  • integrated with LLVM from start to end - a "clang for Onyx". Fast builds - Fast binaries.
  • also still being able to render vanilla c++ source for compilation with other build chains like gcc, avr-gcc, etc. to continue enabling targeting basically any system there is a C++ compiler for (basically any system at all).
  • ofcourse (no surprise) asm.js-ES can be targeted via this backend through emscriptem, etc.
• The "for convenience" useful ones: The dynamic-lang backends
  • Lua-backend (for scripting apps, fast shell-scripts, JIT-compilation)
  • ES-backend (for nodejs, rich web apps, etc.)
    • Also asm.js via above mentioned emscriptem.
  • These will primarily be made to conveniently be able to write Onyx style for these environments
  • The same promises and contracts on math ops, etc. as C++ is not given here. Most things are UB (undefined behaviour)
  • Realistically the un-intrusice strong type system will be a great help in getting better diagnostics and writing better code faster.
  • You use the standard landscape of libs of the target (whether Lua, NodeJS-ES, web-ES or whatever) — just use a better language against them.

[Sod-Almighty]

all available languages suck in one way or another

Damn straight. That's why I'm here.

Ruby and Lisp are AWESOME, but neither are suited for high-performance or static compilation. And C is a complete arse to use. Etc etc ad nauseum.

tl;dr

I dislike that term. Way I see it, if someone can't be arsed to read the damn thing, they should go home.

Be fully usable replacement for C++

Including static compilation with no dependencies.

I was overly optimistic that [Crystal] would be open to good suggestions

They do seem very resistant to new ideas. Which is one reason I no longer play in their yard.

C++ should be the "family" (it's not it, but its' syntax I've loathed)

Actually, with the advent of "C++1y" (closures, move semantics, etc), it's almost usable now. Get rid of those bastard header files, replace templates with generics, strongly type std::function, and allow monkeypatching of classes.... and C++ might yet crawl out of the dark ages and into the light of modern day!

I won't hold my breath though.

Statically linked self-contained musl based binaries

This!

You use the standard landscape of libs of the target (whether Lua, NodeJS-ES, web-ES or whatever) — just use a better language against them

There's a project called Haxe that targets multiple backends. I'd be satisfied with just C++, if it achieved my requirements of [a] zero-dependancy binaries, and [b] not being an utter bastard to use.

Anyhow....what I got out of your above post was essentially "drop the Crystal backend, and suddenly we start making progress at a sensible pace again". If so, I'm all for it. Hell, I'd be prepared to code a large part of the stdlib for you. Provided it wasn't in C.

I'm good at coding. I'm just not good at coding compilers. Give me a language that's nice to use - like Ruby - and I can code lots of stuff.

[ozra]

You sum it up well.

Funny you bring up Haxe, that's an example of what to avoid somwhat, they've made a rather elaborate stdlib-on-top to abstract to multiple systems. But when I think of it; I'm not opposed to it really. I'm confusing objectives here. I'm just not keen on spending time on one — since I myself would target a backend and play with its' libs and features specifically then.

The only onyx-core-stdlib parts will be the basic "always used types" which I think are good to abstract in a minimal-stdlib (Str, Int, ´32, U64, Byte, Map, Set, List, etc. etc.) and stdout/debug funcs (say, _dbg, etc.)

With "lenient" mode code-policy (default unless specific code-policy specified) any core-libs will be helpfully automatically included when definition from them are used (no fixed prelude like in Crystal I reckon)

Regarding C: I have no intention of making a C backend. Only C++. I would be fully open to include one provided whoever PR's that would take responsibility of maintaining it.

The few things (as I've imagined) in stdlib would mostly be mappings from ox-types to backend-types (well, obviously to no types for Lua and EcmaScript) and say, _dbg fn's.

Onyx will promote using strong numeric types with semantic meaning rather than intrinsic "hardware types":

-- Good:
op *(u Volt, i Ampere) -> Watt  -- ignoring AC vs DC for example
   Watt{ u.value * i.value }

power = 48V * 45A

-- Bad:
power = 48 * 45

It won't be forbidden or anything of course. Using non-coercing types will just be Onyx-culture idiomatic recommendation.

I must be clear on the fact that I'm still up to my brain-arse in shit to do — so this chosen path won't magically make development go rocket just yet I'm afraid. But it will not hurt (bar the initial re-write time...)

[Sod-Almighty]

48V * 45A

Yes, +1 for unit typing. Although explicit casts must be available; and the creation of a unit type should be as terse as possible.

I don't know what a brain-arse is, but I'm fairly sure I don't have one ;)

[ozra]

• Yeah, I have suffixes (suffices?) implemented in current Onyx with the syntax described in #77. The unwillingness in Crystal to include a using declarator was sad in that it makes it hard to include special suffixes for just a portion of DSL, Sci or whatever code. With c++ backend that become possible too. The syntax for the body code of the suffix definition will probably change a little.
• Yes! Explicit casts will of course be the way to juggle between types, which inevitably always ends up having to be done for one reason or another.
• Easily auto-creating common op's! Yes! Definitely! The idea currently is simply that the previously removed "type-def-interpreter"-code connected to specific super-types (similar to on inherited macros) should be brought back into Onyx again. By inheriting a specific type (in the example I wrote NumBuilder or so; that's just a suggestion taken out of the air):
  • My idea so far is — just like enum and flags types has a "builder/interpreter" that accepts a list of costant names that are expanded to an incremental enumeration or doublings — so the NumBuilder takes a list of really terse descriptions of operators and the types they take, and the overloadings are generated automatically. The superior thing about this is that you easily for instance disallow Area to be multiplied by Area, Volt by Ohm, pointers to be added (only Offset can be added), etc. etc. which gives an enormous additional help in not making to-tired-to-crack mistakes.
  • Something along the lines of:

type MyNum < NumBuilder
   *, +, -, <=> MyNum, OtherAcceptableNumT, YetOne
   /, **, % MyNum  -- apparantly only wanted this with self here
   / YetOne  -- but div also accepts type YetOne

   init() -> here goes regular typedef code
   foo() -> and so on

That is:

• multiple operators can be listed, along with multiple rhs-types
• should be simple to define overloads for the reverse type order also
• if the exact same op-signature is defined more than once by the patterns, it will be a warning, or if stricter code-policy is declared for the specific file / scope it will be an error.

I don't know what a brain-arse is, but I'm fairly sure I don't have one ;)

Consider yourself lucky! X-D

[Sod-Almighty]

Looking good. Always been a fan of unit types.

Is a brain-arse where brain-farts come out?