zenscript
zenscript copied to clipboard
keean
Why typeclasses?
Is less verbosity the only main benefit of typeclasses over just inputting a set of functions?
Typeclasses can do where bar is a member of Function:
foo(x: A) where Function<A>
bar(x)
Without typeclasses we could do:
foo(x: A, y: Function<A>)
y.bar(x)
I understand @keean proposed other functionality for the where clause, but as for the main functionality of providing pluggable (i.e. dependency injection) interfaces, what is the big benefit that justifies creating a new language?
I think the question is why not just pass all the interfaces as arguments? You can of course do this, after all type classes are implemented as implicit dictionary arguments.
The advantage of type classes is that they are implicit, hence you have shorter more readable function signatures. They can also be inferred, so you don't have to explicitly type the constraints. You don't want to have to explicitly thread y through every function call where you pass type A because you might want to call bar at some point. Further when you realise you need to call bar on a value of type A you don't want to have to refactor every call in the program to pass an implementation.
Thanks for the reply. I appreciate those advantages. I am just thinking about near-term priorities. I think achieving the "perfect" language is a major endeavor that needs a significant amount of funding and time. I've been thinking that your goal is maximum genericity (i.e. the ability to write some code that can parametrized over many types and uses cases) and minimum explicitness (although in other cases you argue for more explicitness where I argue for less).
My near-term goal is not that. That is far too much for me to think about and digest at this point. I am wanting some improvements on JavaScript so I can go use the same code base both on the client and the (Node.js) server. For example a cryptographic library.
- promises (ES6 has this)
- anonymous unions (TypeScript has this)
- guards or match (TypeScript has this)
- integer types (u8, u16, u32, 64)
- structures (handles all the boilerplate on top of
TypedArrays andArrayBuffers) - everything as an expression
- Python-like code block indenting instead of curly brace delimited blocks
I've been thinking about achieving a much simpler transpiler to TypeScript that can meet those near-term objectives.
Later can experiment with moving to higher levels of genericity and less explicitness, if warranted by the use cases that are prominent. And with more funding and momentum behind the initial simpler transpiler.
Further when you realise you need to call bar on a value of type A you don't want to have to refactor every call in the program to pass an implementation.
That works if you only allow every data type to implemented only one way for each typeclasses, but as we discussed in the past (with @skaller) that has limitations also. Otherwise, you could get unintended implicit functionality. So explicitly passing has its benefits.
Edit: Dec. 4, 2017
- integrated low-level C-like coding with the high-level language and memory allocation mgmt (aka GC)
@shelby3 wrote:
@shelby3 wrote:
Perhaps we will eventually think of some low-complexity abstraction that gives us non-marshaled integration for tree data structures between HLL and LLL, and perhaps we can even prove borrowing and lifetimes statically. But I suspect we can not without heading towards the complexity of C++ and Rust.
@keean wrote:
I would not write a web service in 'C' or even 'Rust' ever, this is really ideal ground for a garbage collected language. I would use Node (TypeScript), Django(Python) or 'Go'.
@keean wrote:
My view is there needs to be a high-level-language and a low-level-language that share enough memory-layout and data-structure that data can easily be passed backward and forward.
@shelby3 wrote:
I am wanting to move away from such clusterfucks of never ending aggregation of complexity.
@shelby3 wrote:
My quest in @keean’s repository has to been to get the advantage of GC and perhaps some other aspects of FP such as first-class functions and closures in a simple language that also can do some low-level things such as unboxed data structures. To further erode the use cases where C/C++ would be chosen, to relegate that clusterfucked C++ language tsuris (and perhaps Rust also but the jury is still out on that) more and more to the extreme fringe of use cases. For example, It is ridiculous that I must employ a cumbersome Node.js Buffer library to build an unboxed data structure.
@shelby3 wrote:
[…] so that code which would otherwise be written in for example TypeScript and C/Emscripten would not have to be rewritten later in Lucid. More optimum compiled targets could be developed such as WebAssembly which leverage the full power of the design laid out above, including more optimum compilation of the C-like code. One of my test use cases would be to translate my Blake C code to Lucid.
Regarding one implementation per class, that is the advantage of type classes :-) If you want more you need to pass an object or function (because you want the implementation to depend on the value of the object not the type).
To me it seems you can do nearly everything you want in typescript, so why not use it? I am working on a project in typescript at the moment, and for the ES6 stuff plus types it's good enough.
Regarding goals, I agree maximum genericity is my goal, and minimum boilerplate. However I also want to prioritise readability and understandability over conciseness, as you spend more time debugging and maintaining code than we do writing it. However I am happy for explicitness to be optional, which includes type-inference (type annotations optional) typeclass constraint inference (so constraints/bounds are optional). The only place we need types would be modules, where we want a stable interface that enables separate compilation.
Regarding one implementation per class, that is the advantage of type classes :-) If you want more you need to pass an object or function (because you want the implementation to depend on the value of the object not the type).
If one universal implementation for each universal class was an absolute advantage or even feasible, then we wouldn't need scoping for (type) name conflicts in programming. It is impossible to guarantee one global implementation for the universe, because total orders don't exist.
I am not arguing that it can not be a feature that is desirable in some cases in a partial order.
To me it seems you can do nearly everything you want in typescript, so why not use it?
I guess you didn't read my list above, or didn't realize that the items I listed which TypeScript doesn't do are extremely important for the work I need to do.
However I also want to prioritise readability and understandability over conciseness
Ditto myself. I take this even further than you as I am thinking all types shall be explicitly declared, except in an assignment where either the destination or source is explicit, but not required for both. Since passing function arguments is assignment, then anonymous lambda functions can be implicitly typed by the function's argument type.
I thought typescript included ES6 features, and has integer types as well as strucures (classes). The only one it does not have is "everything is an expression" I think, so it seems close to what you want?
Regarding type classes, it is clear that each function can only have one implementation for any given set of argument types. In other words we can only define one implementation for (+)<Int, Int>
Regarding type classes, it is clear that each function can only have one implementation for any given set of argument types.
Disagree (and my reason was already stated). But I don't want to repeat that argument now. I have said all I want to say about it for now. It isn't a priority issue for me at the moment. We can revisit that later if ever get to the point where I want to implement a "typeclass" language.
Edit: we had a long discussion about this in another thread in 2016.
I thought typescript included ES6 features, and has integer types as well as strucures (classes).
Afaik, there are no integer types in ES6. Are you thinking of 64-bit math methods in ESNEXT?
http://kangax.github.io/compat-table/esnext/
Interfaces and classes do not pack into ArrayBuffers. Perhaps you were thinking of the following proposal which has gained no traction:
http://wiki.ecmascript.org/doku.php?id=harmony:typed_objects
In 2016, I wrote some very complex (untested) JavaScript code to improve on that concept and provide a polyfill. But it has some drawbacks. I need to revisit that code to refresh my memory.
I looked again at Nim, but it is getting too far from JavaScript (wants to be a systems language competing with C, C++, Rust, and Go). For example, lose JavaScript's Promises, ArrayBuffers, etc..
And it doesn't offer 64-bit integer support on JavaScript.
And I'd much prefer the JavaScript output is recognizeable, so the JavaScript debugger can be effectively used.
I think I envision a robust way to transpile from a syntax and type system I want into comprehensible TypeScript. For example, I can use TypeScript's getters and setters to access the fields by name of a DataView on an ArrayBuffer. When not packed into a binary ArrayBuffer, 64-bit integers can be represented with TypeScript's tuples.
It may be slower than normal JS objects due to packing issues, or it may be faster due to non-string property lookup. It certainly seems the right way to process binary network data, but I would probably use native JS objects everywhere memory layout is not important.
but I would probably use native JS objects everywhere memory layout is not important.
Agreed. I didn't intend to imply otherwise. Not only network data, but file data. You know I am working a blockchain design.
Edit: Also memory data. With a blockchain, you need the UXTO stored in RAM and this becomes huge.
nodejs can mmap files from JavaScipt which would work well with this too. You don't have too many choices in-browser, you are limited to Blobs with a streaming interface, and the file IO API is not supported by all browsers.
nodejs can mmap files from JavaScipt which would work well with this too.
Right agreed. Interacting with binary data in JavaScript on Node.js is via very verbose APIs. I'd prefer a language native solution.
You don't have too many choices in-browser
I wrote "client side" but that doesn't mean we are limited to the browser. I am thinking Android and iOS.
Most in the blockchain realm use a systems programming language with good binary control, such as C or C++. But this is painful in other aspects. There is a lot of verbosity that isn't always needed. Need complex libraries such as Boost for asynchrony, etc... Do not have GC by default, etc.. I think it is overkill.
I am searching for a better balance. Rapid coding with less needless verbosity, yet still be able to handle binary data and integer math without verbose APIs. And also some (explicit) typing for correctness checks and better readability (open source).
Others have integrated C/C++ with JavaScript via Emscripten separating out code that requires that control from that can be done more elegantly in JavaScript, but this IMO isn't ideal.
Most code doesn't need to be that heavily optimized (or at least not the first priority). I am thinking there should be something in between a toy language such as JavaScript and a full blown systems language such as C++.
node-webkit could be an option then (npm nw). You can still compile typescript ( for example: https://basarat.gitbooks.io/typescript/content/docs/quick/nodejs.html )
node-webkit could be an option then (npm nw).
Good point and agreed if Node.js APIs are needed client-side.
Well if you want to read and write binary files, its pretty much required. In a browser, FileReader ( http://caniuse.com/#search=FileReader ) is well supported, but FileWriter is not ( http://caniuse.com/#search=FileWriter ).
Well if you want to read and write binary files, its pretty much required.
Obviously Node.js isn't the only way to read and write files on Android and iOS (and even can access it via JavaScript in a WebBrowser instance). Whether one wants to carry all the baggage of node-webkit just for file APIs is not a certainty. But yeah, it is apparently one of the options.
My focus in on the notion that we shouldn't be forced to use a systems language just to do integer operations and packed binary structures. That we should be able to combine these capabilities with a rapid-coding paradigm that has GC, simple-as-pie asynchronous programming, and less optimum but less painful ways to for example manipulate strings (because we don't always need that to be highly optimized).
C++ is too tedious and overly detailed when in most cases we don't need that. Yet we shouldn't have to revert to C/C++ just to get integer operations and packed binary structures.
Premature optimization often means projects that don't get finished.
Yeah performance is battery life and that is important on mobile. But projects that get to market are more important than projects which don't.
For Android and iOS there are things like Cordova, is that the sort of thing you are thinking about?
Yeah there are also Nativescript and Tabrisjs.
But I am not sure if I choose those, because may also want to think about desktop apps as well.
This is a wide open field of experimentation, because we also have convergence of the laptop/desktop and mobile on the horizon perhaps...
Using HTML5, JavaScipt and IndexedDB you can already write software that runs across mobile, pad, laptop, desktop, and big-TVs. It works pretty well, except for annoying corner cases, like iOS limiting IndexedDB to 50MB of storage. It's definitely fast enough for a lot of applications, and HTML/CSS makes good looking UI reasonably easy.
Writing to the least common denominator controlled by gatekeepers who squat on and stall innovation seems to me to be a losing paradigm. What about SD cards, cameras, various multi-touch swipe gestures, etc... Consumers care about features not about how you achieved it.
Well you have to write your own "drivers" for each platform. On Android you have to write in Java to call all the native APIs, but you can call Java you have written from JavaScript if you create an android application shell in Java. Same for iOS (9+) where you can call native APIs from objective-c or Swift, and you can call your own objective-c or Swift code from JavaScript, so you would need a different objective-c or Swift wrapper app. These wrapper apps need to get into the respective app-stores, which means getting Google/Apple to approve your code. You can distribute outside of the app store on both devices, but it requires the user to jump through hoops (android you need to enable a device setting to allow non-app-store apps to install, iOS you need to "trust" the developer of each non-app-store app in the device settings).
Well you have to write your own "drivers" for each platform. On Android you have to write in Java to call all the native APIs, but you can call Java you have written from JavaScript if you create an android application shell in Java. Same for iOS (9+) where you can call native APIs from objective-c or Swift, and you can call your own objective-c or Swift code from JavaScript, so you would need a different objective-c or Swift wrapper app.
Agreed and I was fully aware of that. But good you've summarized for readers.
These wrapper apps need to get into the respective app-stores, which means getting Google/Apple to approve your code. You can distribute outside of the app store on both devices, but it requires the user to jump through hoops
These companies are trying to create walled gardens and be gatekeepers. Me thinks the free market is going to route around these Coasian costs. They are trying to carefully position themselves with "safety" and "convenience" to acquire monopoly control.
In fact, that is one of my other longer-term goals.
If you have an open source app which acts as a base for all other apps and doesn't abuse as a gatekeeper, then it could alleviate this problem. Once everyone has that base app installed, then it takes care of overriding all these toll bridges. Toll because the costs are paid but indirectly.
(android you need to enable a device setting to allow non-app-store apps to install, iOS you need to "trust" the developer of each non-app-store app in the device settings).
Clicking Ok for a confirmation dialog is not problem for users. It is when they make the enabling action modal and hidden, that the achieve insidious control.
Popping up a red alert, "many other users have reported issues with this app" (with a link off to details) would be a sufficient free market solution to safety.
Even Google realizes that ads can't sustain the company. So they are going to have to become monopolistic. Open source should slay this, as it did to Microsoft Windows.
Android defeated iOS in global installed units (not dollar weighted) marketshare because it was more open. Ditto to Android if they make it difficult for users to have freedom.
As I assume you know, Android still lags (or perhaps near parity with) iOS on a dollar weighted basis (such as $ spent on apps), because iOS simply worked better. For example the egregious latency with Android audio was unacceptable for some scenarios and many people aren't even aware of issues like this. Apple's cathedral-style engineering integration was superior, but eventually bazaar-style open source catches up. Takes a while.
@keean I am not actively coding again yet (only the 19th day of my 8 week intensive 4-drug phase of TB treatment)
I find this comment to be relevant:
Maybe what Rust really needs is fewer programmers writing Rust crates and features, and more programmers who use those features to write end-user programs that won’t be redistributed.
This.
The Rust community looks, from the outside, a bit ingrown and subject to a kind of “can you top this (complexity)” syndrome. I don’t mean this observation in a hostile way; if I were eyeball-deep in the development of a young language I would probably behave the same way – and arguably I did, back in Python’s earlier days when I was briefly on that devteam.
But yes, I think more feedback from people who just want to get stuff done and have a lower complexity tolerance would be good for the language.
Complexity is even more insidious than that, I’m sad to say. It’s not enough to avoid being excessively clever. Even if you restrain yourself, complexity will still creep on you anyway. You need to actively fight it whenever your can, and periodically review your “complexity budget.”
Rust has already, in my opinion as a production user, overdrawn the complexity budget in one place. This happened accidentally, very early on. Specifically, it’s the interaction between several Rust features: (1) The auto-deref features provided by
Deref. (2) The split between “owned” types likeStringand reference/slice types like&str. (3) The unification-based type inference that allows you to writelet i: u32 = "10".parse()? and automatically choose the right version ofparse. (4) TheIntotrait (and related traits), which allows you to write functions that take arguments of typeInto<String>, which essentially means, “Oh, just give me any type that advertises conversion to aString.”Any one of these features individually makes Rust much nicer to program in. But if several of them all gang up in the same piece of code, a novice Rust user will get burnt. If I’m reading between the lines correctly, this might actually be what bit you with
bind. The bind function takes an argument of typeToSocketAddrs, which is basically one of thoseInto-like traits which allow you to pass in several different ways of representing a socket address. It’s cute and clever (yuck), and it works most of the time. But if it gets combined with (1-3) above, it’s going to generate crappy error messages. The fix is relatively simple for an experienced Rust programmer, if annoying: Add explicit type declarations all over the place. But at work, my guideline is “just don’t useInto-style conversion traits in your APIs unless you have a damn good reason. They don’t actually improve API ergonomics.”If this is the only place where Rust exceeds it’s complexity budget, well, users will just learn this one bad interaction and go about their lives. C++ has dozens of warts worse than this. But any further expansions of Rust in this area need to be handled very carefully, and production Rust users need to participate in the RFC process.
But let’s dig deeper.
There are two libraries in the Rust space which worry me: Diesel and Tokio. Diesel looks like an ORM, but it’s really not—it’s just a typed version of the relational algebra which can dump output into Rust data types. It results in really concise and efficient code once it’s working. But the error messages are just horrendous (though not yet in modern C++ territory, though that’s nothing to be proud of). Diesel has chosen to push Rust’s type system to its limits in the name of speed and expressiveness. I’m not sure it’s worth it. We had a long debate at work and I paired on Diesel code with one of our other Ruby backend guys, and he said the tradeoffs with Diesel’s error messages were worth it. I’m not 100% sold.
Where I’m more concerned is
tokio. As everybody has told you,tokiois central to the Rust async I/O story, and lots of popular crates are moving totokio-based backends (though most will still export synchronous APIs). And from what I’m hearing,tokiois currently generating bad error messages for some common use cases. In my opinion, this needs to be fixed—and the core team is discussing pushing up a language feature that’s been in the RFC process for a while now, which will hopefully make the error messages much clearer.Still, I’m holding off on
tokio-based async I/O for at least 6 months in production code, to see how this all plays out.
And the comments complaining about the JavaScript ecosystem chaos such as this one:
The situation with JavaScript and Node.js is more complex. At the time of writing, there are 399,773 packages on npmjs.com. And yet, there are really obvious categories with no good alternatives. For example, I needed a CLI option parser the other day, and I needed it to support git-style subcommands. I searched npmjs.com for 30 minutes and spoke to our in-house Node guru, and everybody said, “No, every single Node package for your use case is terrible.” This makes me sad and frustrated. I eventually found something that was, frankly, pretty bad and cudgeled it until it worked.
And as I had pointed out on the Rust forum back in 2016, all the complexity doesn't really provide us anything that really helps us that much because most programming errors are not constrained to the sort of safety Rust is checking for (yet those safety checks are very debilitating to code in), as reiterated by this comment:
What you described is probably true for a lot of user-space code, but it is not true for the kernel. If you look at Linux kernel CVEs, you will see that they have nothing to do buffer overflows, stack smashes, use-after-free vulnerabilities. They are more likely to be logical errors, which can be exploited under some condition. Even when you hear a data race condition found in the kernel, it is unlikely to be caused by a missing lock. In most cases, it is a logical error of some kind. For example, here is the patch for a recently discovered data race in Linux (CVE-2016-5195): https://lkml.org/lkml/2016/10/19/860. As you see, it was not about locking, but about proper checking different flags.
Moreover, Linux developers actively use a static analyzer to catch the most common mistakes in their code. So Rust can’t help there much. Also, it is completely unrealistic to write a general purpose kernel without unsafe code. Even if you look at the Rust standard library, you will find a lot of unsafe code there (e.g. all collections). When you work with hardware at low-level, you have to do a lot of things that can be potentially unsafe. But let’s suppose that it was possible to write the whole kernel in a safe programming language, would it eliminate security exploits? Of course, not.
You may look at long list of CVEs for many web-frameworks written in safe languages (such as PHP, Python, Ruby). By its nature, the kernel works at a higher privilege than the user-code, which means many logical errors in the kernel can be exploited to do something that the user should not be allowed to do. So writing a safe kernel is a far more complex task then writing a kernel in a safe language. First of all, you have to have a formal definition of what you mean by “safe” and then decide how you are going to prove that. It is very tricky even for a very small toy kernel.