elks
elks copied to clipboard
ghaerr
New Cool Stuff for ELKS
I'm opening this issue to talk about various projects I'm working on, most of which are running, but not fully integrated into the ELKS build yet. I'm interested to know how interested people might be in each. Here's the first three projects:
- ELKS remote console display
- D-Flat Text UI windowing toolkit
- Debug stack traces for ELKS executables
I'll write up each of these separately.
ELKS Remote Console
This little program allows one to login through a serial or telnet connection to ELKS (using a macOS or Linux terminal program, for instance), and display what is currently displayed on the ELKS console. This could be useful to see remote kernel messages, or just see exactly what is displayed in full color, using the ROM BIOS 437 character set.
For example, here's a login from macOS Terminal over emulated serial on QEMU, just after boot:
The display is updated every second, and the program is terminated with ^C.
The cons program reads the PC text RAM contents directly and converts the characters into UTF-8 encoded Unicode values, along with interpreting the PC character attributes (color, reverse video, etc) and translating those into ANSI escape sequences.
The current implementation just displays the console screen, it doesn't try to insert keystrokes, etc.
D-Flat Text User Interface Library
D-Flat is an old (1995 era) text user interface that allowed overlapping window-based text user interfaces to be written on DOS. I've ported it to UNIX/Linux, and now to ELKS. It is very well written, but unfortunately kind of large, and requires medium model (large code) for compilation. The underlying API is message-based, and very similar to Win32, with SendMessage and window procedures.
Here's a screenshot of the D-Flat memopad program, which allows for editing files:
I'm primarily working on it as a possible TUI for the Cosmopolitan project, but thought it would be interesting to get it running on ELKS.
Now to show off the cons program more, here's the cons program running on serial macOS Terminal, while memopad happens to be running on the console:
Debug library for ELKS applications
While trying to get larger ELKS programs working, I've found it quite hard to understand what is going on when the program crashes, especially with no debugger or stack tracer. I've developed a set of routines that will allow for a running ELKS program (currently applications only, but could be fitted into the kernel itself for device drivers etc) to produce a stack backtrace, along with correct lookup of the actual C symbol names.
I've also implemented Cosmopolitan-style C function logging capability, which allows for display of every C function executed, to make it possible to visualize function call flow throughout program execution. I plan on adding nanosecond-precision timing display (on 386+ CPUs only, using the RDTSC instruction) to show function latencies as well. The idea would be to add an automatic "--ftrace" program argument that would turn on this display when an application is compiled for debug.
I took ELKS basic and added the stack trace capability on every function call, here are the first few functions called, along with their arguments. The display also shows the current and max stack used by the program:
ELKS 0.6.0
login: root
# basic
AX=0000 BX=7460 CX=0001 DX=7464 SI=7460 DI=3448 BP=7446 SP=7422
CS:IP=5540:36f0 DS=6200 ES=6200 SS:SP=6200:7422
Level Addr BP DI SI Ret Arg Arg2 Arg3 Arg4
~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0: 7418 => 7446 3448 7460 36f7 dead 0000 0000 0000 0000 0000 0000 0000 0000 0000 print_stack (364e,83)
1: 7446 => 744e 2c60 7460 3448 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 __cyg_profile_func_enter+35 (36f7,83)
2: 744e => 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 0000 7478 7482 748d main+b (3448,83)
3: 745c => 0000 0001 7472 0000 7478 7482 748d 7497 74a1 74b2 0000 6162 6973 0063 _start+19 (0019,81)
4: 0000 => 0000 0000 5825 0900 2800 2900 2b00 2d00 2a00 2f00 3d00 3e00 3c00 3c00 _start+1 (0001,81)
>main, from _start+19, stack 0/0
AX=0019 BX=0000 CX=5401 DX=0000 SI=7460 DI=2ce5 BP=7414 SP=73f0
CS:IP=5540:36f0 DS=6200 ES=6200 SS:SP=6200:73f0
Level Addr BP DI SI Ret Arg Arg2 Arg3 Arg4
~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0: 73e6 => 7414 2ce5 7460 36f7 dead 00a1 0019 73fe 73fe 354e 7408 1742 1c9c 5014 print_stack (364e,83)
1: 7414 => 7442 0000 7460 2ce5 0000 3448 0000 0000 0000 735f 6174 7472 312b 0000 __cyg_profile_func_enter+35 (36f7,83)
2: 7442 => 744e 0000 7460 345b 0000 46b8 745c 2c60 2cb0 0019 0001 7460 7464 0000 tty_isig+b (2ce5,83)
3: 744e => 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 0000 7478 7482 748d main+1e (345b,83)
4: 745c => 0000 0001 7472 0000 7478 7482 748d 7497 74a1 74b2 0000 6162 6973 0063 _start+19 (0019,81)
5: 0000 => 0000 0000 5825 0900 2800 2900 2b00 2d00 2a00 2f00 3d00 3e00 3c00 3c00 _start+1 (0001,81)
|>tty_isig, from main+1e, stack 25/25
I am also thinking of a use where a function backtrace would be displayed whenever ^C (SIGINT) is typed, to show what the program is doing at the time of interrupt. This would be very useful for debugging hung programs.
The instrumentation facilities used to implement function call tracing require a patch to the ELKS compiler toolchain, and the stack trace capabilities require some special link handling which adds debug symbol information into the current executable, both of which are rather complicated at the moment.
In addition, I have some library code that can interpret memory bytes and display as 8086 disassembly, running directly on ELKS. This could be used within an application, or added to the hd or a new disasm or monitor/debug program, for better visibility of very low-level stuff while running on ELKS. The hd program is going to be enhanced to display codepage 437 (ROM BIOS) characters on a terminal display, to better see binary values as part of a hex dump.
Very interesting indeed, @ghaerr. I strongly support the ^C stack trace idea, and by extension - the ability to send a process a signal to have it display a 'current' stack trace to the console. Which in turn would make the cons program (neat idea!) immediately invaluable.
Apropos nanosecond resolution - don't you see a challenge in the way the output would affect the timing?
Disasm: Sounds very useful, but I didn't understand how it may be used. Are you thinking like 'hd -disasm /dev/mem' (actually some part of /dev/mem)?
Thank you - this is great work!
--M
Debug library for ELKS applications
While trying to get larger ELKS programs working, I've found it quite hard to understand what is going on when the program crashes, especially with no debugger or stack tracer. I've developed a set of routines that will allow for a running ELKS program (currently applications only, but could be fitted into the kernel itself for device drivers etc) to produce a stack backtrace, along with correct lookup of the actual C symbol names.
I've also implemented Cosmopolitan-style C function logging capability, which allows for display of every C function executed, to make it possible to visualize function call flow throughout program execution. I plan on adding nanosecond-precision timing display (on 386+ CPUs only, using the RDTSC instruction) to show function latencies as well. The idea would be to add an automatic "--ftrace" program argument that would turn on this display when an application is compiled for debug.
I took ELKS basic and added the stack trace capability on every function call, here are the first few functions called, along with their arguments. The display also shows the current and max stack used by the program:
ELKS 0.6.0
login: root
basic
AX=0000 BX=7460 CX=0001 DX=7464 SI=7460 DI=3448 BP=7446 SP=7422 CS:IP=5540:36f0 DS=6200 ES=6200 SS:SP=6200:7422
Level Addr BP DI SI Ret Arg Arg2 Arg3 Arg4
0: 7418 => 7446 3448 7460 36f7 dead 0000 0000 0000 0000 0000 0000 0000 0000 0000 print_stack (364e,83) 1: 7446 => 744e 2c60 7460 3448 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 __cyg_profile_func_enter+35 (36f7,83) 2: 744e => 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 0000 7478 7482 748d main+b (3448,83) 3: 745c => 0000 0001 7472 0000 7478 7482 748d 7497 74a1 74b2 0000 6162 6973 0063 _start+19 (0019,81) 4: 0000 => 0000 0000 5825 0900 2800 2900 2b00 2d00 2a00 2f00 3d00 3e00 3c00 3c00 _start+1 (0001,81) >main, from _start+19, stack 0/0 AX=0019 BX=0000 CX=5401 DX=0000 SI=7460 DI=2ce5 BP=7414 SP=73f0 CS:IP=5540:36f0 DS=6200 ES=6200 SS:SP=6200:73f0 Level Addr BP DI SI Ret Arg Arg2 Arg3 Arg4 ~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 0: 73e6 => 7414 2ce5 7460 36f7 dead 00a1 0019 73fe 73fe 354e 7408 1742 1c9c 5014 print_stack (364e,83) 1: 7414 => 7442 0000 7460 2ce5 0000 3448 0000 0000 0000 735f 6174 7472 312b 0000 __cyg_profile_func_enter+35 (36f7,83) 2: 7442 => 744e 0000 7460 345b 0000 46b8 745c 2c60 2cb0 0019 0001 7460 7464 0000 tty_isig+b (2ce5,83) 3: 744e => 745c 2c60 2cb0 0019 0001 7460 7464 0000 0001 7472 0000 7478 7482 748d main+1e (345b,83) 4: 745c => 0000 0001 7472 0000 7478 7482 748d 7497 74a1 74b2 0000 6162 6973 0063 _start+19 (0019,81) 5: 0000 => 0000 0000 5825 0900 2800 2900 2b00 2d00 2a00 2f00 3d00 3e00 3c00 3c00 _start+1 (0001,81) |>tty_isig, from main+1e, stack 25/25 I am also thinking of a use where a function backtrace would be displayed whenever ^C (SIGINT) is typed, to show what the program is doing at the time of interrupt. This would be very useful for debugging hung programs. The instrumentation facilities used to implement function call tracing require a patch to the ELKS compiler toolchain, and the stack trace capabilities require some special link handling which adds debug symbol information into the current executable, both of which are rather complicated at the moment. In addition, I have some library code that can interpret memory bytes and display as 8086 disassembly, running directly on ELKS. This could be used within an application, or added to the hd or a new disasm or monitor/debug program, for better visibility of very low-level stuff while running on ELKS. The hd program is going to be enhanced to display codepage 437 (ROM BIOS) characters on a terminal display, to better see binary values as part of a hex dump. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.
Hello @ghaerr ,
Thank you for very cool stuffs! I started to port nano-X to pc-98 but it seems harder than I thought so it would be nice to have the debug library.
D-Flat is also interesting to see applications like DOS edit on ELKS but I'm not sure it will work for PC-98 too.
Hello @tyama501,
I started to port nano-X to pc-98 but it seems harder than I thought
It may be easiest to just copy nano-X/drivers/scr_bios.c to scr_pc98.c, and plan on having all graphics draw code in that single file. Rename all the functions from VGA_ to pc98_, and then start with just writing the pc98_open() and pc98_close() routines which should switch the screen to and from graphics mode, using the same code from BASIC.
You should then be able to write the very basic pc98_drawpixel, pc98_drawhline, pc98_drawvline and pc98_fillrect functions, which do not depend on an internal graphics format. (HAVEBLIT should be set to 0).
After that, in order to support BLIT, we would need to make a decision as to how nano-X should store the graphics data internally, which is set with the psd->app and other statements in pc98_open().
If you'd like to share what you have, I'm happy to help you get the screen driver working. The mouse driver, but default uses the serial port, which isn't yet implemented for PC-98, so you might want to compile in drivers/mou_null.c.
D-Flat is also interesting to see applications like DOS edit on ELKS but I'm not sure it will work for PC-98 too.
What is the character set for byte values 128 - 255 in PC-98 character ROM? These won't be codepage 437, so that could require low-level changes to D-Flat for running on console. However, I am thinking having the library code automatically switch to using ANSI ESC sequences when running on telnet or serial, and having the CP 437 codes converted to unicode for that case (the same was cons works, basically).
Thank you!
Hello @Mellvik,
Apropos nanosecond resolution - don't you see a challenge in the way the output would affect the timing?
Yes, the trace and timer code will definitely add overhead to all function tracing, but the idea is it will be a fixed amount, and will adjust all timings by that same fixed amount, so bottleneck latencies can still be easily spotted. The routine could calculate its own overhead and subtract that from the displayed time for better accuracy (I've added that thought to my list).
Disasm: Sounds very useful, but I didn't understand how it may be used.
I think the most useful case would be some kind of debug/monitor that would trap to a prompt (interrupting the running program from a SIGINT), then producing a backtrace and allow the user to dump register or specify a seg:off memory address to start disassembly from. As a library routine, it would also be useful to write programs to disassemble various files at times. Another use could be instruction tracing, by allowing a single instruction to execute on real hardware, followed by an INT 3 back to the monitor. This would work because the disassembler has the ability to compute how long a given instruction sequence is.
I'm not quite sure where to put all this - I suppose just adding the the C library would be OK, except these are all non-standard routines. So perhaps a separate library... not sure yet.
I'm also working on being able to compile and link programs outside of the ELKS source tree - this is too hard currently, as there's no easy way to get all the compiler options and C library. Once this is done it will be lots easier to contribute programs to ELKS, without having to become intimately familiar with the kernel and application source trees.
Thank you!
Thank you @ghaerr -
Disasm: Sounds very useful, but I didn't understand how it may be used.
I think the most useful case would be some kind of debug/monitor that would trap to a prompt (interrupting the running program from a SIGINT), then producing a backtrace and allow the user to dump register or specify a seg:off memory address to start disassembly from. As a library routine, it would also be useful to write programs to disassemble various files at times. Another use could be instruction tracing, by allowing a single instruction to execute on real hardware, followed by an INT 3 back to the monitor. This would work because the disassembler has the ability to compute how long a given instruction sequence is.
Wow, that's ambitious indeed!! I'm rather looking forward to it - exciting!
I'm not quite sure where to put all this - I suppose just adding the the C library would be OK, except these are all non-standard routines. So perhaps a separate library... not sure yet.
My 2cents - separate library.
I'm also working on being able to compile and link programs outside of the ELKS source tree - this is too hard currently, as there's no easy way to get all the compiler options and C library. Once this is done it will be lots easier to contribute programs to ELKS, without having to become intimately familiar with the kernel and application source trees.
Agreed, and again very useful indeed!
ELKS rocks
-M
Hello @ghaerr
It may be easiest to just copy nano-X/drivers/scr_bios.c to scr_pc98.c, and plan on having all graphics draw code in that single file.
I have copied vgaplan4.c to vgaplan4_pc98.c and I have been modifiying ega_drawpixel to draw directly to 0xA800, 0xB000, 0xB800, 0xE000 VRAM after changing to the graphic mode from basic but your idea sounds easier for the first step.
the serial port, which isn't yet implemented for PC-98, so you might want to compile in drivers/mou_null.c.
I have noticed this, and I commented out GdOpenMouse in srvmain.c for now. OK, I will use mou_null instead.
For PC-98 there is no serial mouse, and the mouse is attached to 8255 I/O port. So, maybe we need to write the driver with polling or interrupt after the screen driver worked.
D-Flat is also interesting to see applications like DOS edit on ELKS but I'm not sure it will work for PC-98 too.
What is the character set for byte values 128 - 255 in PC-98 character ROM?
I poked to 0 - 255 directly to text vram with basic :) It seems there are single byte katakana-characters and some others.
Thank you.
Hello @tyama501,
I have copied vgaplan4.c to vgaplan4_pc98.c and I have been modifiying ega_drawpixel to draw directly to 0xA800, 0xB000, 0xB800, 0xE000 VRAM after changing to the graphic mode
I see. I suppose it depends on how close the PC-98 graphics hardware is to EGA - EGA is terrible, and requires multiple mask and plane register to be set for each pixel. If that's the case for PC-98, perhaps staying with your present approach is better. Another thought would be to only support mono (2-color) mode initially, is that possible, then possibly not having to write to each memory plane for each pixel.
After getting running, you will want to "define NDEBUG" to remove the assert() code, as it slows things down considerably.
For PC-98 there is no serial mouse, and the mouse is attached to 8255 I/O port. So, maybe we need to write the driver with polling or interrupt after the screen driver worked.
When you use mou_null, this should happen automatically. Afterwards, a PC-98 mouse driver can be written.
In general, nano-X busy-loops by polling, so this shouldn't be a problem unless the polling itself is quite slow.
It seems there are single byte katakana-characters and some others.
I can see it's definitely not codepage 437, but does include the box line draw characters. There doesn't seem to be the 50% gray shaded character that is used for the background of D-Flat windows, but that isn't a big problem, a space can be used.
Should we get more interested in actually running D-Flat on PC-98, we can develop a translation table for the different code page, both for D-Flat use, as well as for the cons unicode translation.
Thank you!
Hi @ghaerr,
I am for D-Flat project - it seems it can produce a nice DOS editor and some other applications as well. We can have a developer section one day: "Programming for ELSK" using:
- pure C code
- using X-nano
- using D-Flat TUI
- BASIC
D-Flat and its text-mode user interface library demo is now running on ELKS connecting from serial or telnet terminal sessions. Here's a video showing it in operation. In this particular case, all I/O is based on ANSI terminal escape sequences, so the TUI display is available to many remote systems, using any modern terminal emulator, such as macOS Terminal or Linux xterm-256.
It also runs on the console, but this requires more setup as a mouse driver that spits out ANSI sequences needs to be installed, unless the program can be entirely operated from the keyboard.
The video is too big for Github upload, here's a link to my dropbox for viewing.
Nice job @ghaerr !!! So it can run natively (without any remote sessions) on ELKS, but one needs to use the keyboard shortcuts.
D-Flat and its text-mode user interface library demo is now running on ELKS connecting from serial or telnet terminal sessions. Here's a video showing it in operation. In this particular case, all I/O is based on ANSI terminal escape sequences, so the TUI display is available to many remote systems, using any modern terminal emulator, such as macOS Terminal or Linux xterm-256.
It also runs on the console, but this requires more setup as a mouse driver that spits out ANSI sequences needs to be installed, unless the program can be entirely operated from the keyboard.
The video is too big for Github upload, here's a link to my dropbox for viewing.
Wow @ghaerr - this is impressive.
Apropos mouse: Can the interface be operated w/o a pointing device?
-M
- jul. 2022 kl. 07:14 skrev Gregory Haerr @.***>:
D-Flat and its text-mode user interface library demo is now running on ELKS connecting from serial or telnet terminal sessions. Here's a video showing it in operation. In this particular case, all I/O is based on ANSI terminal escape sequences, so the TUI display is available to many remote systems, using any modern terminal emulator, such as macOS Terminal or Linux xterm-256.
It also runs on the console, but this requires more setup as a mouse driver that spits out ANSI sequences needs to be installed, unless the program can be entirely operated from the keyboard.
The video is too big for Github upload, here's a link to my dropbox for viewing https://www.dropbox.com/s/qlssrndb007vjwa/D-Flat%20ELKS%20Demo.mov?dl=0.
— Reply to this email directly, view it on GitHub https://github.com/jbruchon/elks/issues/1366#issuecomment-1186776237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3WGOD73SAHWWVG744YM5DVUTR5BANCNFSM53JQFRHQ. You are receiving this because you were mentioned.
Can the interface be operated w/o a pointing device?
Yes, I'm sure it could, but not certain as to what has been coded in D-Flat. D-Flat is a somewhat large text user interface (TUI) library, by ELKS' standards. The memopad demo, which includes all the TUI controls (scrollbars, listboxes, popups, comboboxes, edit boxes, etc) is 110k - huge. D-Flat is IBM SAA/CUA compliant as of their 1987 standard.
However, the real purpose of the work is mostly R&D, and under the hood portions of the following are already implemented, or being considered:
- Working medium model ELKS applications (a toolchain bug was recently fixed getting memopad to load properly).
- Writing TUI applications that compile and work almost identically between macOS, Linux and ELKS.
- Unicode support, which allow ELKS to run applications that can display unicode (using UTF-8) on serial or telnet connections. In particular, the ability to display BIOS character ROM codepage 437 characters external to the console is a big win.
- Possible solutions to long-standing character display problems such as #1175 because the PC ROM character set isn't the same across machines internationally.
- Enhancement of incoming and outgoing ANSI escape sequence support for ELKS console and terminal sessions, including accepting a wider ranger sequences on direct console for UNIX application compatibility, as well as ANSI mouse and window sizing support being put in library routines.
- A possible ability for an application to know whether its running on direct console or serial/telnet terminal, and to automatically perform character conversions via Unicode and/or ANSI sequences for visual compatibility. (For instance, the
consconsole display program, and plannedhddisplay enhancements to see the full CP437 character set). - Possible support for arbitrary unicode on non-console applications. This is important, as the rest of the world is now running on unicode, and ANSI sequences allow unicode display on all modern terminal emulators.
- Development of multiple layers of TUI support, from automatic mouse/keyboard support without termcap or ncurses (using ANSI/xterm standard) for building portable TUI applications, the ability to write characters/attributes to the screen directly for speed, all the way to heavier-weight libraries like D-Flat.
- Test programs that allow for easier unit-testing of various library components.
- Testing of the kernel and C library implementation by using larger pieces of software (more incompatibilities have been found).
All this said, the fact that ELKS is basically running almost all of D-Flat identically to its UNIX counterpart is a testament to how far ELKS has come in the last few years - its pretty amazing our 16-bit system can keep up and run more modern UNIX 32- and 64-bit programs, thanks to everyone's work :)
@ghaerr Are you planning on submitting D-Flat as part of the bin folder? Maybe only for the HDD images?
Hello @toncho11,
My plan would be to consider adding the D-Flat source code as a text user interface library, along with a number of other helpful library routines, somewhere in the ELKS source tree. The library would be automatically built so that others could build their own TUI applications. The D-Flat library includes only a single demo memopad that displays all of its capabilities which would also be copied to the /bin directory for any images 2.88M and larger.
The real problem with most TUI and GUI libraries is that it isn't trivial to build an application using them. Frankly, it isn't real clear to me what anyone might build with a D-Flat TUI. (Given that the kilo and nano editors don't work well, we renamed the MINIX screen mined to edit, and it seems to work quite well, so we don't need a screen editor in D-Flat). Even though D-Flat is well thought-out and well-written, it is large by ELKS standards, and programs produced using the library won't fit well on floppy disks with the rest of the standard distribution.
Thus, D-Flat itself is mostly a research project, which (described above) is helping to illuminate some things that might want to be considered next for development. That said, it's opening the doors for more discussion on what we should next add to ELKS. Thank you for your comments!
Again - an impressive undertaking @ghaerr, and - as you point out - an important demo of how far ELKS has come. It is indeed impressive what can be tweaked out of limited resources with smartness, patience and attention to detail (read: avoid the bloat risk). I got a vivid demonstration of this the other day when I had to boot W95 on a 386sx in order to check out some PnP issues. Even at 40MHz and w/16MB mem W95 is totally useless while the resources are overkill for ELKS (feels like QEMU). Not a fair comparison, but a reminder. ELKS is (or can be) one floppy, W95 is 31 floppies to get started :-).
However, the real purpose of the work is mostly R&D, and under the hood portions of the following are already implemented, or being considered:
An important angle - and ambition: Create a platform for new ideas, experimentation rather than being a small scale copy of larger siblings. Or simply creating a tool for some special purpose. My 2C along that angle of thinking:
Working medium model ELKS applications (a toolchain bug was recently fixed getting memopad to load properly). Important, resource utilization (btw - your recent work on debugging infrastructure falls into this category). Writing TUI applications that compile and work almost identically between macOS, Linux and ELKS. Unicode support, which allow ELKS to run applications that can display unicode (using UTF-8) on serial or telnet connections. In particular, the ability to display BIOS character ROM codepage 437 characters external to the console is a big win. Agreed - UTF-8 is an important step. Possible solutions to long-standing character display problems such as #1175 because the PC ROM character set isn't the same across machines internationally. Enhancement of incoming and outgoing ANSI escape sequence support for ELKS console and terminal sessions, including accepting a wider ranger sequences on direct console for UNIX application compatibility, as well as ANSI mouse and window sizing support being put in library routines. Important - and challenging. Reliably reading multibyte ANSI sequences is touch on both serial and telnet, even on Linux and RaspberryPis with vastly more resources than ELKS. And frequently evidenced in ELKS and vi today - on serial. A possible ability for an application to know whether its running on direct console or serial/telnet terminal, and to automatically perform character conversions via Unicode and/or ANSI sequences for visual compatibility. (For instance, the cons console display program, and planned hd display enhancements to see the full CP437 character set). Possible support for arbitrary unicode on non-console applications. This is important, as the rest of the world is now running on unicode, and ANSI sequences allow unicode display on all modern terminal emulators. Agreed, immediately very useful. Development of multiple layers of TUI support, from automatic mouse/keyboard support without termcap or ncurses (using ANSI/xterm standard) for building portable TUI applications, the ability to write characters/attributes to the screen directly for speed, all the way to heavier-weight libraries like D-Flat. Test programs that allow for easier unit-testing of various library components. Testing of the kernel and C library implementation by using larger pieces of software (more incompatibilities have been found). Indeed. Beating up the system in practical usage is when the 'deep' bugs pop up. The buffer bug I'm on right now being a great case in point. All this said, the fact that ELKS is basically running almost all of D-Flat identically to its UNIX counterpart is a testament to how far ELKS has come in the last few years - its pretty amazing our 16-bit system can keep up and run more modern UNIX 32- and 64-bit programs, thanks to everyone's work :)
A final point: We (the world) are heading into rough waters these days, and will need to become more resource conscious. To me ELKS is part of that. Industrial control systems in the thousands that were deemed obsolete and useless years ago can do serious work with better platforms - like ELKS. The 386sx system mentioned above is a great example: a 20+ years old SBC perfectly useable for a lot of purposes with the 'right' ('unbloated') software. Taking that angle, ELKS development or more than a hobby, it's a contribution to sustainability.
Thanks again, this is great work.
--M
Thus, D-Flat itself is mostly a research project, which (described above) is helping to illuminate some things that might want to be considered next for development. That said, it's opening the doors for more discussion on what we should next add to ELKS. Thank you for your comments!
Considering D-Flat, in regards to as what should be added next - probably, a simplistic lightweight twin-panel file manager, based on D-Flat?
Considering D-Flat, in regards to as what should be added next - probably, a simplistic lightweight twin-panel file manager, based on D-Flat?
That's a nice idea, but potentially a lot of work to (re)write a nice file manager. What would help would be something like a 16-bit version of Midnight Commander, which was discussed a bit in #1117, but apparently not available. With the D-Flat lower level ANSI/xterm input/output, we could eliminate the need for the huge and clunky ncurses library, and only use xterm/ANSI sequences, which would display properly on ELKS and most all Linux/macOS systems. That is, an older ncurses-based file manager could be slimmed down and possibly more easily ported by replacing ncurses with the D-Flat lower level.
Does anyone have any ideas about an older, small file manager that might be portable?
Looking around for other "small" text-based file managers, I finally found two, both based on ncurses: nnn (https://github.com/jarun/nnn) which is highly supported and does lots of stuff. It purports to running well on embedded systems. Here's a screenshot:
However, in reading a comparison article, nnn apparently uses far less memory than other Linux-based file managers, "only" 3.5MB of RAM!!! (compared to 9.8MB for Midnight Commander, and 51MB for the largest):
The nnn executable on macOS is 131K with no options enabled.
I guess RAM is cheap.
Looking further, I find noice (https://github.com/RichardHyde/noice), which it turns out nnn is forked from. It compiled quickly on macOS, and is a 29K executable. Its development stopped in 2015, but it looks very similar, just doesn't use any color display:
The nnn source is 8800 lines, and noice is 1000. Looking at noice, it seems it could be ported to ELKS quickly, with an initial version using just the arrow keys (and/or mouse on a remote terminal), using the low-level D-Flat framework I just developed. Later, a version could be possibly inserted into a D-Flat frame to look cooler. I haven't looked at the memory usage yet for noice. I may take a pass at porting noice to ELKS if people are interested.
I looked further into noice and found the original project (https://git.2f30.org/noice), which was last updated in 2019. It is a quite small and nifty, portable file manager.
I ported this to ELKS as our new fm file manager, and wrote an ncurses library emulation layer for it, which allowed fm to run directly on the console or from a serial/telnet terminal. The file manager is only 13K bytes, so it should run well on floppies.
Here's a screen recording of it in operation. I plan on porting the lower layer to be able to use the mouse or function keys on the console or serial/telnet terminal, and then ultimately should be pretty straightforward to put into a D-Flat window. Pretty cool!
https://user-images.githubusercontent.com/11985637/180363966-87eef0b8-ae6e-4f26-a284-e87f93935bce.mov
Thanks for the idea @ATroubleshooter :)
Indeed @ghaerr - this IS cool!
- jul. 2022 kl. 06:49 skrev Gregory Haerr @.***>:
Here's a screen recording of it in operation. I plan on porting the lower layer to be able to use the mouse or function keys on the console or serial/telnet terminal, and then ultimately should be pretty straightforward to put into a D-Flat window. Pretty cool!
-M
Update on new File Manager for ELKS and UNIX:
Significant progress has been made, the the project now compiles and runs identically on ELKS console, ELKS terminals, as well as UNIX (tested currently only on macOS Terminal). It seems to work pretty well, and for now have created its own project, before integrating directly into ELKS.
Here's a screenshot running on macOS (which pretty much looks identical to the ELKS terminal version):
The file manager has lots of features for its small size, but here are the basic keystrokes:
RightArrow/Enter - Enter directory or display file
LeftArrow/Backspace - Back up one directory
Up/DownArrow or mouse scroll wheel - move cursor
a-z - goto file matching letter pressed
Z or ^C - quit
D - directories displayed first
T - sort by time
S - sort by size
C - change directory
E - edit file
. - display hidden files
~ - home directory
? - help
The program automatically decompresses .Z files for display, and will run man on man page files in compressed or normal format. more is used to display files.
After cloning the source tree, type make for the Linux version, or make -f Makefile.elks for the ELKS version, then type ./fm.
I've worked hard to make the program as small as possible, which is pretty small since ncurses and regex libraries were removed. fm requires and ANSI (or xterm-256) terminal to run.
The ELKS version is 20k and UNIX 40k. Let me know what you think. At some later point I may wrap the output in D-Flat to look more like Midnight Commander.
Enjoy!
I see edit is possible. Here are suggestions for future versions:
- configure editor (which is the default now?)
- copy files with progress bar
- delete and move both files and directories
Looking back I like Norton Commander, but PC Tools is the one that impressed me the most back then. PCtools is not double screen. A new window pops up and you can select directory tree or drive (I think).
configure editor (which is the default now?)
vi is the default, but any editor can be specified using the EDITOR= environment variable.
copy files with progress bar delete and move both files and directories
Good ideas, but fm is currently just single-paned. I'll wait and see whether people actually use it before spending too much time on it. With the under-the-hood work on the keyboard and mouse input for D-flat, it was really good timing to work on another program that heavily used the mouse and arrow keys. D-flat doesn't yet support the mouse wheel, and I'm going to jump back into that for bit and try to get a version others can play with on ELKS or UNIX.
Update on D-Flat for ELKS and UNIX:
A separate project has been created for D-Flat, where it can be compiled for macOS, Linux or ELKS. Things have been progressing nicely, and it now automatically sizes to the open Terminal on startup, just like the fm File Manager. Both projects share the same new "unikey" ANSI keyboard/mouse library, which also supports unicode character conversions (currently only for code page CP437). The screen updates are FAST.
Here's D-Flat's memopad running on macOS. The display looks identical when run from ELKS serial or telnet:
The new unikey library supports ANSI mouse sequences from macOS and Linux terminal programs emulating xterm-256, and I plan on adding the mouse sequence support to ELKS running QEMU over an emulated serial port. That would allow for operation of QEMU console programs using the mouse from your host computer or trackpad to operate it. On real hardware, the serial mouse is already supported, but not yet built into the unikey library, which would allow for a new class of programs to be developed for ELKS - text based user interfaces which could be sped up greatly just by using the mouse wheel for fast scrolling, for instance.
If you have time, take a look at the project and compile it. Because both D-Flat and File Manager are being developed for both UNIX and ELKS, I'm thinking that adding them as a Git submodule (i.e. a repo within the ELKS repo) in the future, which would allow for automatic compilation and inclusion on the ELKS disk images.
Hello @ghaerr ,
Nice! Yes I agee adding other projects as submodules to expand ELKS world. It would be nice if more developers contribute to ELKS through those projects.
Thank you.
With the console fixes in PR #1388, ELKS can now simulate the Matrix! :)
https://user-images.githubusercontent.com/11985637/182286487-20db97ca-94bb-4e60-b168-7c109295c9ea.mov