Added libffi bindings

[dimkr]

trafficstars

I added libffi bindings:

1. They allow the use of native functions and system calls that don't have Tcl wrappers (pretty much everything except the basics already provided, like fork() and kill())
2. Sometimes when you decide to write complex logic in C because performance is crucial, running some parts on top of a very lightweight interpreter (such as Jim) is handy: it provides stuff like exceptions and garbage collection, without much overhead
3. I'm currently working on a FOSS security software pet project: one of its components is a client that receives dynamically-generated scripts from a server, then evals them; I decided to implement it using Jim and libffi, instead of using C and temporary shared objects (or some other ugly solution), because these scripts do low-level userspace stuff

(Re-opened #32, after rebasing)

[dbohdan]

Ping. This is a potentially very interesting feature that few interpreters have out of the box. Are there plans to merge it?

[msteveb]

Yes, sorry. I had a quick look and meant to give a full response. The interface seems a bit cumbersome. How does it compare to the Tcl ffi interface? Is it close enough to the same to be compatible

[dbohdan]

Yes, sorry. I had a quick look and meant to give a full response.

Ah, good to hear you're considering it.

The interface seems a bit cumbersome. How does it compare to the Tcl ffi interface? Is it close enough to the same to be compatible

The interface is different from Ffidl's. I haven't really looked at other libffi bindings for Tcl.

PYK's example on the wiki page demonstrates what code that uses Ffidl looks like in practice. Personally, I think I somewhat prefer Ffidl's API (especially ffidl callout to ffi::function) but I am not sure it is significantly less cumbersome. LuaJIT's FFI is, AFAICT, widely considered state-of-the-art. A short while ago I reimplemented the same example with it for comparison.

[evanhunter]

@dbohdan & @dimkr - I have just rebased this on my test-everything branch and tried it, and it fails under test immediately due to objects leaked from Jim_ffiInit():

-------------------------------------
Objects still in the free list:
0x25521c0 (0) variable  : '::one'
0x2551fd0 (0) variable  : '::zero'
0x2552210 (0) variable  : '::null'
0x2552630 (0) variable  : '::main'
-------------------------------------

To test, I suggest you grab my branch https://github.com/evanhunter/jimtcl/tree/ffi and run: ./tools/ci-test/ci-test.sh ./tools/ci-test/extra_tests.sh

These run all sorts of extra tests including valgrind, static analysis, various sanitizers etc. You could also use the code coverage tool to ensure your ffi tests are adequate.

Evan

[msteveb]

Thanks Evan. I think the approach here still needs some work but I will take a while to write up my specific concerns.

[dimkr]

Right now I'm not sure if preparing this feature for upstreaming is worth the effort, considering 1) the fact @msteveb disagrees with the high-level design 2) the generally slow development and contribution flow of jimtcl 3) how busy I am and 4) the little motivation I have to spend time on this pull request after the creation of https://github.com/dimkr/szl and its (IMHO, much more elegant) FFI bindings.

However, @evanhunter - if you want and need to get this pull request merged, I'll do whatever I can to help, even beyond of the scope of this pull request.

[msteveb]

I think the goal should be to make as simple as possible to create Jim Tcl extensions for an API. I would probably rather favour something along the lines of critcl (https://wiki.tcl-lang.org/page/Critcl)