Github Actions

[Torinde]

Would be good to have automatic builds / Github actions - like https://github.com/crazii/SBEMU/pull/21.

A combined package with SBEMU would also be useful, but I don't know if you can do that across repositories (e.g. USBDDOS action to get the latest artifact from SBEMU-main and SBEMU action to get latest artifact from USBDDOS-master)

@volkertb, can you help with some of that?

[volkertb]

I can look into that. But the README mentions "cross toolchain on linux not tested". There are ways to build in DOS in GitHub Actions Worklows. I created a container image that was meant for such workflows. But it would complicate the pipelines, and likely make them slower too.

Being able to cross-compile and build the binaries from Linux (in addition to being able to build them from within DOS) would be beneficial.

[crazii]

I'll add a makefile for Linux, and test it on MinGW. BTW, I remember there was a free-to-use BCC cmdline toolchain (without IDE) release by Borland, it could be used to build the BC edition. I'll try to add another makefile for that too.

UPDATE: the BC toolchain is win32 only, not working on DOS. besides I cannot find a valid download link, maybe it could be found on webarchive but that site is again off limits for me. EDIT: found one on here: https://osdn.net/projects/sfnet_udev/downloads/OldFiles/freecommandLinetools.exe/

[volkertb]

I know you are the primary developer and putting the vast majority of the work in, but I would like to implore you to try not to be dependent on toolchains that include any closed source components, especially if they are abandonware. In my opinion, software isn't truly "Free" if it depends on any non-Free components for it to be built. Some may consider that a form of purism, but to be considered for inclusion in open source distributions such as FreeDOS, this is important as well.

Also, from a purely practical sense, it's better when the build tools that we depend on are also still being maintained and improved.

Personally, I would consider the ability to also be buildable on those older closed source compilers and toolchains (in addition to the primarily supported open source ones) to be a nice bonus at best.

And also for the purpose of CI/CD, open source toolchains are a much better choice, especially when this allows the DOS binaries to be cross-compiled from a Linux environment.

[crazii]

Either you got me wrong or I'm not sure what's the point. I totally agree with you on free softwares.

As said in the readme, the project initially was planned to build through DJGPP, OpenWatcom, Borland C. There're 2 reason that the Openwatcom is abandoned: 1) DOS4G/W or DOS32A has limit DPMI support, which still needs a HDPMI host to get full DPMI support. 2) 32 bit mode doesn't have 16 bit inline assembly.

There're also 2 reason that BC was included, which is not mentioned in the readme: 1.The original project was built using Turbo C, but that's not really important 2.The most important thing (for me) is that it's just FUN. I might be emotional involved easily when talking about Borland C, because I was a big fan of Borland. I still remember the happy coding times with TC++/BC++ that Borland brought me like DOS games. Retrogaming is fun, but it's also true for retrocoding.

There's no need to be 'a bonus at best' because it just DOESN'T HURT, because there's a DJGPP built that's being worked on at the same time. Using Borland C is also practical for debugging; DJGPP is too slow on an old P3 PC and is too slow on modern PC when boot and compile on USB-disk, but BC is way faster than that (tens of times faster or more), and saves a huge amount time for me. compile on modern PC and pull & plug on the testing PC is also wasting too much time.

There's also a possibility that OpenWatcom is brought back, but in real mode, which drops DPMI support and uses the same approach as in BC. But I'm focused on the EHCI now and too busy to test & try that. it will be planned. But even with Openwatcom working, I'm not dropping BC personally.

[Torinde]

FYI, but probably you're aware already:

OpenWatcom is in active development - maybe you can make an Issue/PR there about the missing functionality?

There are also JWasm, Jwlink, HX-OpenWatcom-Debugger and forks thereof (e.g. jwasm.info)

[crazii]

Created. https://github.com/open-watcom/open-watcom-v2/issues/1178

[volkertb]

@crazii It looks like I indeed misinterpreted your earlier post. Thanks for your clarification. I understand your fondness of Borland C, and the practical benefits it brings to have it supported as well.

I was just reminded of other open source projects that do exclusively depend on closed source toolchains (notably SoftMPU), which made me worry about it here as well, but that's not justified.

[volkertb]

@crazii @Torinde PR created. Please have a look: https://github.com/crazii/USBDDOS/pull/6

NOTE: with the most recent (Watcom-related?) changes, DJGPP builds started failing again. I tried fixing them, I could not immediately figure it out, and I wanted to have the PR out first, since it was working locally right up until the last few upstream commits were pushed.

[volkertb]

@crazii I just realized, if you are indeed going to try supporting (Open) Watcom in addition to DJGPP and Borland C, I could add a workflow to automatically test builds with Open Watcom as well as DJGPP. That way it is immediately noticed when any pushed changes breaks compatibility with either of those two compilers. What do you think?

[crazii]

I'm currently working on the OpenWatcom build, there's progress, but freezes on some real mode call, I didn't take care of the DJGPP compiling yet, there might be failure, I can fix them now, and I will finally check it when the OpenWatcom is ready. BTW the DJGPP makefile can work under Linux and DOS, and the OpenWatcom will be the same.

[crazii]

I just pushed the fix for DJGPP. I'll check the PR later, maybe when the Openwatcom is done.

[volkertb]

Thanks! After rebasing on these fixes and fixing a few more things in the workflow, everything appears to be working now in my PR.

The workflow now successfully builds the EXE with DJGPP and publishes it as a release. :+1:

Two remaining minor items before this PR can be merged (other than it having to be reviewed):

• I still need to uncomment a line in the workflow so that the publish job will only run on commits and merges to the default (master/main) branch. I had it commented out to test that job while I was working on it.
• Instead of TODO, we might want to place some other text or template in the release notes. Just let me know what to put there.

Thanks again!

[crazii]

Cool. TODO is not a release notes but a dev memo, are you talking about CHANGLOG? it simply record change histories. Share you idea on release notes. And there's still some to be done, i.e. the exe should show a version string and a probably copy right string.

[crazii]

BTW you can test the binary with your Mac Mini, the EHCI driver might works, I've tested it on two laptops and it works fine. If it doesn't you make build a debug edition (make DEBUG=1) to check the logs on the screen.

[volkertb]

Thanks. I'll try that when I get back home. Currently visiting family.

By the way, I've just tried testing it on a different machine, and it failed. Although it bears mentioning that this was on an installation with SBEMU already loaded. (This installation is based on the SBEMU USB image that is automatically created on each release by GitHub Actions, so including Jemmex, HDPMI32i and SBEMU.) Perhaps that isn't supported yet?

The command hang for a while, then gave an error and showed a command prompt again, but it was no longer responsive to keyboard input, and the system had to be restarted.

See the screenshot below:

I also retried this without lh, but that didn't make any difference, and it crashed in the same way.

I still have yet to try it with a debug-enabled build, as you suggested, or without having SBEMU loaded first.

To give you more info about the CPU and motherboard chipset of this system:

Output of cat /proc/cpuinfo:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 16
model           : 6
model name      : AMD Athlon(tm) II X2 250 Processor
stepping        : 3
microcode       : 0x10000c8
cpu MHz         : 800.000
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate vmmcall npt lbrv svm_lock nrip_save
bugs            : tlb_mmatch apic_c1e fxsave_leak sysret_ss_attrs null_seg amd_e400 spectre_v1 spectre_v2
bogomips        : 6027.05
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 16
model           : 6
model name      : AMD Athlon(tm) II X2 250 Processor
stepping        : 3
microcode       : 0x10000c8
cpu MHz         : 800.000
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate vmmcall npt lbrv svm_lock nrip_save
bugs            : tlb_mmatch apic_c1e fxsave_leak sysret_ss_attrs null_seg amd_e400 spectre_v1 spectre_v2
bogomips        : 6027.05
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

Output of lspci:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] RS880 Host Bridge
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] RS780 PCI to PCI bridge (ext gfx port 0)
00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] RS780 PCI to PCI bridge (PCIE port 2)
00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] RS780/RS880 PCI to PCI bridge (PCIE port 5)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 3c)
00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 IDE Controller
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor HyperTransport Configuration
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Address Map
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Link Control
01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 430] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF108 High Definition Audio Controller (rev a1)
02:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

[volkertb]

@crazii By the way, I just realized it might be useful to have GitHub Actions automatically generate the debug-enabled variant as an additional release artifact as well, with a suffix like _dbg behind the file name just before .exe, to distinguish it from non-debug-enabled artifact. That would allow anybody to easily download and try the debug-enabled build for each release.

The executable is small, so also offering the debug-enabled variant beside it is no big deal.

I'll update the PR for this.

[volkertb]

Update: I tried it again on the same AMD system mentioned above, this time replacing LH SBEMU with LH USBDDOSP /HID in the most recent SBEMU USB image, with everything else being the same. I got the exact same result. (Again, the error wasn't shown until after it froze for a couple of seconds.) Interestingly, not only did the keyboard stop working after the error was shown, but disk access (to C:) stopped working after the failed launch of USBDDOSP as well.

It may be useful to know that the motherboard of this older AMD system already has a PS/2 port, a combined one for both keyboard and mice that requires a splitter. I only had a USB keyboard and a USB mouse hooked up to it, though. But maybe the presence of an actual PS/2 interface might conflict with the utility? Perhaps it was written with systems completely lacking PS/2 ports in mind? I find that unlikely, since you've been testing the tool on a couple of laptops, including some older models, which I believe still come with (combined) PS/2 ports as well, right?

See the screenshot below.

[volkertb]

@crazii Alright, I ran into something weird.

Building without DEBUG=1 works fine both on my current development machine (using act to test the GitHub Actions Workflow locally), but fails in actual GitHub Actions, with the following error:

output/emm.o:emm.c:(.text+0xcf): relocation truncated to fit: 16 against `.text'
output/emm.o:emm.c:(.text+0xf4): relocation truncated to fit: 16 against `.text'
collect2: error: ld returned 1 exit status
make: *** [Makefile:49: output/usbddosp.exe] Error 1
Error: Process completed with exit code 2.

I committed (and later reverted) a temporary change that disabled the debug-enabled build variant (so it would only build the non-debug-enabled variant) and that one worked fine, also in GitHub Actions.

This is really weird, since with act, the GitHub Actions run is simulated locally by running all the jobs and steps in the same Docker images.

Note that currently, when running the workflow with act, it will fail later on in the publish-job, due to Node.js missing, even though that job will run fine in GitHub Actions (whenever build-job succeeds). But that's an unrelated known issue. Right now, the relevant issue is that the same compilation of the debug-enabled variant passes locally, yet fails on GitHub, with seemingly the same parameters and Docker images.

The only factor that I can rule out is differing hardware in the GitHub test runners compared to the local machine that I'm testing this on right now. (The aforementioned AMD system, with 8GB of RAM.)

Can you maybe look into this, when you're ready to take a look at this PR? Thanks.

[crazii]

There isn't much info, only error initializing the device connected on port 0. it'd better test with a debug version.

The usbddos failed, it's expected that the USB flash disk will not work because usbddos take ownership from the BIOS, so the BIOS emulation stopped working anymore. If you're booting from a USB flash disk, then /disk parameter is essential to re-enable flash disk, but this time the drive is handled by usbddos instead of BIOS emulation.

Maybe an continuing on error could help this situation a little bit so that error on other ports/devices won't affect the port of USB flash disk. but still, if error occurs on communicating with the USB flash disk, the drive won't work anymore.

[crazii]

I searched a little bit, no clues about the error, except that you may try to add -mxgot in the CFLAGS & CXXFLAGS

[volkertb]

Thanks, I'll try that. In the meantime, I created a separate issue (#7) for that AMD system that I just tried to test USBDDOS on. I also uploaded a video recording of the debug-enabled build (compiled locally), including all of its debugging output up until it failed. I hope it's useful to you. The M4A88T-M/USB3 motherboard is interesting, since it's a board that included USB 3.0, not as part of the south bridge, but through an integrated NEC USB 3.0 PCIe controller.

[volkertb]

I just tried -mxgot locally, but DJGPP didn't support it. Apparently, this option is available only when compiling for M680x0, ColdFire and MIPS architectures: https://man7.org/linux/man-pages/man1/gcc.1.html

[Torinde]

@volkertb, great! Maybe in a next PR - can you make a combined release USBDDOS+SBEMU? And vice-versa - the Github action in SBEMU to include USBDDOS as part of the builds there.

the exe should show a version string

I suggest showing date and Git commit number - like this:

And the filenames to be also like that, e.g. USBDDOS-2023123123045-GitNumber.zip?

[volkertb]

@Torinde That makes sense, but I'll wait for USBDDOS to mature a bit first, since right now it's hanging on the two different machines that I tried it on.

Have a wonderful New Year (and beyond)! :slightly_smiling_face:

[megatog615]

I am anticipating autobuilds, since I can't seem to get it compiled on my DOS system(make complains about "target pattern contains no '%'" at line 40 of the makefile).

[crazii]

I am anticipating autobuilds, since I can't seem to get it compiled on my DOS system(make complains about "target pattern contains no '%'" at line 40 of the makefile).

which makefile/tool are you using? the default Makefile uses DJGPP. Makefile.bc uses Borland C++3.1

[megatog615]

I am anticipating autobuilds, since I can't seem to get it compiled on my DOS system(make complains about "target pattern contains no '%'" at line 40 of the makefile).

which makefile/tool are you using? the default Makefile uses DJGPP. Makefile.bc uses Borland C++3.1

I am typing "make." I have djgpp installed on FreeDOS and have the environment set up(well, as far as I can tell anyway).

[crazii]

I am typing "make." I have djgpp installed on FreeDOS and have the environment set up(well, as far as I can tell anyway).

I checked, the makefile have problem detecting freedos, I'll try another way.

[crazii]

I am anticipating autobuilds, since I can't seem to get it compiled on my DOS system(make complains about "target pattern contains no '%'" at line 40 of the makefile).

which makefile/tool are you using? the default Makefile uses DJGPP. Makefile.bc uses Borland C++3.1

I am typing "make." I have djgpp installed on FreeDOS and have the environment set up(well, as far as I can tell anyway).

Is LFN working on your FreeDOS? I tried FreeDOS liveCD FD13LIVE.iso on VirtualBox, it seems the doslfn is resident already, but LFN is not working, not long name after the common entries of DIR command. and I modified the makefile but the make will fail for finding a uppercase of the source file.

EDIT: DOSLFN is bundled with FreeDOS but not resident (not installed on default config), running it doesn't have effect.

[crazii]

I am typing "make." I have djgpp installed on FreeDOS and have the environment set up(well, as far as I can tell anyway).

I fixed the problem now LFN is not needed by makefile. But the makefile still doesn't work properly on freedos, the line $(shell dir ..\*.c /s/b): %DIRCMD% default has a '/p' on freedos, which will cause the make wait for std input, after a key press everything compiled but at last there's an error: No rules to make target 'Press', needed by 'output/usbddosp.exe'. Stop. Funny problem. It behaves the same on MSDOS with the same '/p' set on DIRCMD. but Fortunately I don't use DIRCMD at all.

Anyways just using $(shell dir ..*.c /s/b/-p) works for FreeDOS. And I just pushed the change.