temu-vsb
temu-vsb copied to clipboard
volkertb
Make binaries compatible with FreeDOS
It would be nice if the Virtual Sound Blaster could be compatible with the JEMM386 memory manager that comes with FreeDOS. Right now the program simply returns that you need EMM386 or QEMM386 for it to run.
@Build-0-Matic That would be great. But I could really use some help with this. My assembly programming skills are still limited. My first priority has been to make this code build with an open-source assembler, whether that's WASM, JWASM, FASM or NASM. The code isn't truly free as long as it's dependent on closed-source software. Right now, it still only builds properly with TASM.
So far, I've spent quite some time trying to port the code to an assembly dialect that would make it build on those other assemblers, and I've gained experience in doing so, but with my current knowledge and skills I'm not even close to making that work.
Currently, the steps I have in mind are the following:
- Get the code to build with a completely open-source toolchain (assembler and linker)
- Add support for JEMM386
- Add support for additional (more modern) output devices, such as AC'97 ICHx, Intel HD Audio, USB Audio, etc.
- Combine ADLIPT and VSB into a unified emulator that sends FM to an OPLxLPT device in one parallel port, and digital audio to another supported device (a Covox on another parallel port, or one of the aforementioned newer modern output devices)
I believe all of this is doable, but I really need some help from others with this!
Assembly-savvy people I've found on GitHub so far who could perhaps help us out with this:
- @vincentbernat
- @stsp (dosemu2 developer)
- @andrewbird (dosemu2 developer)
- @pdewacht (developer of ADLIPT and other DOS OPLxLPT tools)
- @JKnipperts (forked temu-vsb to create a driver/emulator for the TNDLPT device)
- @Baron-von-Riedesel (a.k.a. Japeth, developer of Jemm386 and JemmEx )
- @jmalak (maintainer of the Open Watcom v2 fork, has been kindly improving JWASM to hopefully make it build temu-vsm)
- @tkchia (does a lot of DOS development stuff, including maintaining a fork of gcc-ia16)
- @PerditionC (FreeDOS kernel developer)
- @vladr (VDMSound developer)
Sorry to mention you guys, but if any of you are willing to help us out with this project, and/or can get us in touch with (more) people who are, that would be greately appreciated.
Long story short: dust off Andrew Zabolotney's Virtual Sound Blaster TSR from the '90s and make it useful again, by...
- ...making it build with open-source toolchains
- ...making it support more modern sound devices
- ...geting it ready for distribution with FreeDOS
Any help is welcome! Thank you. :innocent:
@Build-0-Matic By the way, in the meantime you could try booting into FreeDOS without having any EMM manager loaded. In other words, with FreeDOS running in actual real mode instead of in V86 mode.
The original vsb.com executable is actually a combined executable containing two different .com files, one for when QEMM386 is loaded (assembled from vsb_qemm.asm) and the other for when no EMM manager is loaded at all (assembled from vsb_real.asm).
I understand that this is inconvenient and that many DOS games with which you want to use VSB would work better with access to Expanded Memory. But this way, you can at least test whether VSB still works properly on more modern systems.
I'm curious to hear whether it works for you. Also, if you know of any people with dood DOS x86 assembly skills that would be willing to help us with this project, please get them in touch with me. Thanks. :slightly_smiling_face:
@Build-0-Matic That would be great. But I could really use some help with this. My assembly programming skills are still limited.
You're better off than I am as my assembly skills are nonexistent.
I'll try running it with nothing installed so that I get the most memory.
Keep an eye on this issue: https://github.com/open-watcom/open-watcom-v2/issues/615
Once WASM is cable of building https://github.com/volkertb/temu-vsb/blob/master/sbemu/vsb_qemm.asm, that will be an important step towards tackling this, since people willing to assist in the porting effort from QEMM to JEMM will no longer require the proprietary and DOS/Windows-only TASM assembler to build the code.
Then again, anybody savvy enough to port this from a QEMM QPI TSR to a Jemm Loadable Module (JLM) is probably also skilled enough to port the code from TASM dialect to NASM dialect or some other dialect that can be built with an open-source assembler.
Oh, I also opened a feature request to have QPI support added to JEMM386, which would make it compatible with TSRs that currently depend on QEMM386. Not sure how often @Baron-von-Riedesel has time to check these issues and feature requests, though, much less work on them. See https://github.com/Baron-von-Riedesel/Jemm/issues/6
What are the goals? vsb was a very poor emulator, compatible with a few realmode games, incompatible with dpmi iirc. Do you want an sb over pc-speaker? There are much better ways of doing so.
Thanks for asking!
As far as I know, VSB is the only open source Sound Blaster emulator for DOS that at least somewhat works. (I would gladly be proven wrong on it being the only one, by the way!)
The PC speaker support is obviously not interesting. More interesting is the existing Covox Speech Thing support, the advantage of that being that it's both cheap and would (theoretically) work even in newer post-ISA systems, as long as they still have a DOS-compatible parallel port.
But additionally to the Covox support, I was hoping to use this as a basis on which to build further, for instance by supporting more modern sound output devices.
But you mentioned much better ways of doing so? Could you elaborate?
It might also be useful to emphasize that we are looking for a solution that could be shipped with FreeDOS and would work with an as wide a range of PCs as possible, old and new alike.
For old and new pcs I dont have a solution, no. As a task of replacing vsb, 20 years ago I wrote a pc speaker driver for linux, and an sb emulator for dosemu (no dosbox back then). I wanted to write a civox driver for linux, but didnt have covox. I would suggest fullfilling that gap now and just write a covox driver. Not a solution for old machines, but good for new ones.
VSB already supports Covox. But I wanted to make the code independent from proprietary tools such as TASM and TLINK first. And then gradually improve it from there.
Do you think it would be more feasible to abandon the VSB code, instead take the sb16.c code from dosemu2 and build a Jemm Loadable Module (JLM) around it that traps the Sound Blaster and Adlib ports? And then send the audio output to a Covox device for now, and then add support for other sound devices later?
And would there be any advantage to doing this in assembly language instead of C? The modern systems that don't have ISA slots anymore are usually fast enough for that not to matter, right?
I've just tried it on FreeDOS without anything running from the config.sys file but I still get the QEMM required message. And that is no matter which file I run.
@Build-0-Matic Have you tried vsb_real.com? That one should work without QEMM. If not, then I don't understand why there even are two variants of the VSB TSR.
@stsp I agree, it wouldn't be a solution for every possible case.
But older (pre-386) systems usually have ISA slots and you can just install actual Sound Blaster or compatible cards in them. It's mostly a (kind of) solution for the lack of sound support that FreeDOS offers people who try to run it on more modern systems.
And yes, it would only work with real mode games, at least at first, until Sound Blaster emulation could perhaps be built into a DOS extender like DOS/32A. But that's outside the scope of this project. (It would be a cool follow-up project, though!)
Let's tackle things one at a time. :slightly_smiling_face:
Yes, vsb already supports covox, but adding such driver to linux is not difficult at all. You'll immediately get dos games to use it. As for coding for freedos - my own opinion is that this is not a 21th century approach. Which is why I created fdpp - fully 64bit freedos port that can work on top of linux kernel, getting use of any sound drivers. I do not expect everyone to share that view, but I firmly believe this is the only possible future of the dos. :) (currently fdpp is supplied with dosemu2)
@Build-0-Matic Have you tried
vsb_real.com? That one should work without QEMM. If not, then I don't understand why there even are two variants of the VSB TSR.
Tried it too... And still the same result. It's like if something in FreeDOS itself made VSB think a memory manager was installed.
I run FreeDOS on a netbook off a USB stick. I found a trick for installing it to USB so I don't have to run it in a virtual machine (though it involves installing to VirtualBOX, formatting the USB Stick to FreeDOS Boot using Rufus, opening the VM drive with PeaZip and extracting it's content to the USB.... I know it's not a clean process but it works).
@Build-0-Matic So even vsb_real.com complains about QEMM not being loaded? That's really weird, since the README actually stated that that variant was meant for environments without QEMM. :confused: The README mentions an older version of VSB that didn't require QEMU at all. Maybe you can find it online? Otherwise, perhaps you can inquire at forums such as the DOS forum at VOGONS for further help on this.
@stsp Yeah, I noticed that project too. And by the way, dosemu2 has been working great so far. It's just that I'm fascinated with the fact that native (16-bit) DOS seems to continue to fully work bare-metal even on the latest PCs, with the exception of sound card support. I think that's really the itch we're trying to scratch here.
Of course, I understand that once newer PCs start to become UEFI-only without legacy BIOS support, running 16-bit DOS bare-metal will finally no longer be possible. And then options like fdpp, virtualization and emulation will remain as the only viable ways to run DOS on modern hardware. And in such cases, emulating Sound Blaster within DOS will indeed become pointless, since it can better be handled by the hypervisor or emulator instead of inside DOS.
@stsp By the way (and this is a bit of topic, I admit), would it be feasible to have fdpp run in a dom0 domain on a type 1 (bare-metal) hypervisor such as Xen? So no Linux kernel underneath, but with privileged direct access to VGA hardware, and integrated Sound Blaster emulation? So UEFI -> Xen -> fdpp (dom0) -> DOS?
It would be a modern and reasonably future-proof solution that would be as close to running on bare-metal as possible, while still providing a solution for the lack of Sound Blaster compatibility on modern systems.
I guess the downside of such a solution is that you wouldn't be able to piggyback on the wide range of supported sound devices that the Linux kernel supports.
Ok, if you insist on building it under freedos, then my best guess is that this should be done around hdpmi. Binding it to qemm or emm386, as the original vsb does, will likely leave you short of prot-mode games, as unless dpmi is there, they use vcpi to load their own ring0 kernel with an extender around it. And if you want to do that on C, which is what you most definitely want if you intend to port sb16.c of dosemu2, then you first need to come up with a dpmi extension to trap io ports.
Yes, fdpp is intended to be portable. The main problem is that v86 mode is not available on modern cpus, and I would guess its not available under xen. In which case you still need dosemu2 to emulate v86.
A DPMI extension to trap I/O ports? Seems way above my abilities. Also, would this work for most of the protected mode games, that run on the DOS/4GW DOS extender? Are they compatible with the HDPMI host, even if such an extension were to be written for it?
As for v86 mode on Xen: if I understand correctly, Xen leverages Intel VT-x and AMD AMD-V hardware virtualization extensions, which allow a guest VM to run in 32-bit mode, including V86 support. Isn't this the same mechanism that dosemu2 also uses through KVM to enable native (non-TCG) execution of 16-bit code on an x86-64 host system? Or doesn't this apply in the privileged dom0 domain?
A DPMI extension to trap I/O ports?
Yes, preferably compatible with those of emm386 or qemm (which vsb already uses), to not reinvent the wheel. Having such extensions, you'll be able to port sb16.c using djgpp toolchain.
Are they compatible with the HDPMI host, even if such an extension were to be written for it?
Of course. HDPMI is the one of the most compatible DPMI impls, and most compatible means supports most available extenders.
Isn't this the same mechanism that dosemu2 also uses through KVM to enable native (non-TCG) execution of 16-bit code on an x86-64 host system?
Well, yes, dosemu2 can set up v86 via KVM, and also it can emulate v86 or use JIT to re-compile real-mode code to protected mode. Neither of these approaches is simple enough, but of course KVM is much simpler than jit or emulation. If some glue between fdpp and KVM is written that can work with xen, then fine. But then I suspect that under xen you can't grab all the needed hardware, anyway (like timer, interrupt controller etc). And if you can't grab some hardware, you need to emulate it. So I don't believe fdpp can run under xen directly, but perhaps the needed wrapper can be much smaller than dosemu2 on linux. Some wrapper will be needed.
Btw, KVM under linux emulates all the needed hardware already. If the same is true under xen, then indeed it might be possible to run fdpp there.
In HVM mode, Xen uses QEMU to emulate necessary hardware. Additionally, if fdpp were to be running in the dom0 domain, it would also get privileged direct hardware access to the host machine (useful for things like VGA). Shall I open a separate issue in the dosemu2 or fdpp GitHub projects to discuss this further?
As for your suggestion to write a DPMI extension for port trapping, I'll try to do some more research on it. In the meantime, if you can share any links to documentaiton, or perhaps any GitHub projects doing similar things, I would appreciate it. Thanks.
I never used xen. But I very much doubt you can grab all hardware, like timers and interrupt controller. Wikipedia says the paravirtualized kernel can work in dom0. Also fdpp requires the C/C++ runtime, which is unlikely to be available in dom0 (unless you boot linux there first). Note that you can grab VGA also under linux - dosemu can do that if you disable the linux modesetting first. So VGA access is not unique to xen.
In the meantime, if you can share any links to documentaiton, or perhaps any GitHub projects doing similar things, I would appreciate it.
This should be very easy if the DPMI host runs apps on ring-3 (which I think is what hdpmi does). In that case, you only need to make sure the IO bitmap in TSS is set properly, and you need to install the GPF handler that will be called on every port access. Then you need to decode the access and call the user's handler. This all can be done in C with djgpp. The only DPMI extension you need here, is for the ioperm() call. We can ask @Baron-von-Riedesel if what I say is right, and if so - how difficult is to implement the ioperm() call in hdpmi (but it can't be too difficult).
This should be very easy if the DPMI host runs apps on ring-3 (which I think is what hdpmi does)
Yes
In that case, you only need to make sure the IO bitmap in TSS is set properly
in standard HDPMI, clients run with CPL & IOPL=3, so no I/O is trapped, and the - one and only - TSS has no IO-bitmap. It's possible to create a hdpmi exe that runs clients with IOPL=0. In this version, HDPMI "emulates" the IN/OUT opcodes in ring 0. But there's still no IO-bitmap.
It would be nice if the Virtual Sound Blaster could be compatible with the JEMM386
Jemm386 has an io-bitmap. And there exists a ring-0 extension (JLM) that offers an API to trap port access, just like MS Emm386.
But what if under jemm386 the apps will use vcpi to install their own ring0 dpmi kernel? Which is what I think they do, unless dpmi is already there. Or am I missing something?
@Baron-von-Riedesel Instead of having to port VSB to a JLM, Would it be practical to add QEMM (QPI) compatibility to Jemm386? I opened a feature request for that: https://github.com/Baron-von-Riedesel/Jemm/issues/6
Or am I missing something?
No, that's quite correct. There's a full context switch whenever the pmode app calls v86-mode, so the io-bitmap of jemm386 is relevant only for "real-mode" apps.
Would it be practical to add QEMM (QPI) compatibility to Jemm386
I don't know QPI, but I guess it's too much work involved.
My first priority has been to make this code build with an open-source assembler, whether that's WASM, JWASM, FASM or NASM
jwasm probably would be the best choice. Translating a 16-/32-bit segmented app to FASM would be a hard job.
I looked into the source code and tried to assemble it with jwasm.
There are a few trivial errors ( ENDP and ENDS without preceding name of the proc/struct), and a few data labels (Desc386) have a trailing ":" ( which makes them code labels in MASM/JWASM style ). The most problematic thing probably is that a "mixed" group is created, containing both 16-bit and 32-bit code segments. This is NOT allowed in MASM/JWASM, resulting in error "cannot access label through segment registers". It's not hard to circumvent this, but it requires a rather good understanding of assembly language and the segmented memory model.
No, that's quite correct. There's a full context switch whenever the pmode app calls v86-mode, so the io-bitmap of jemm386 is relevant only for "real-mode" apps.
Which is why my proposal to implement it in hdpmi instead. I don't suppose IO bitmap is difficult to add (isn't it trivial?), and the rest can be done in a user-supplied GPF handler, so no other work on hdpmi side. Don't you like such plan? :)
I don't suppose IO bitmap is difficult to add (isn't it trivial?),
Well, currently the TSS in in conventional memory, so adding an io-bitmap would actually require moving it to extended memory. Additionally, there's no API to access the bitmap from ring 3, so one would have to design and implement one. This makes this job "hard" :))
so adding an io-bitmap would actually require moving it to extended memory.
Just to save 8K of an UMB space? Oh, this can be optional, so no one will really care. :)