FDOS
(E)DR-DOS kernel interprets sector fill value F6 as valid media descriptor byte
I noticed that at least @ecm-pushbx s EDR-DOS and Calderas OpenDOS kernel 7.03 accept an unformatted partition created by Free and MS FDISK as valid FAT-16 volume. My first guess is that the fill value of 0xF6, which FDISK uses to erase the volume boot sector, is interpreted as valid media type by the DR-DOS kernel, and therefore the kernel tries to make sense of the BPB in some form. The volume shows with serial number F6F6:F6F6. MS-DOS kernel interprets F6 as invalid media type, and thus refuses to access the disk.
This is especially annoying in connection with FAT-32 partitions, because the DR kernel, regardless auf the partition type, ALWAYS interprets such unformatted partitions as FAT-16, and FreeDOS format tries to format it as FAT-16 accordingly.
There are three options:
- change Free FDISK
- and/or change EDR-DOS kernel to refuse access on media type F6
- ignore
The DR-DOS DRBIO code path is as follows: in disk.asm, a default bpb is created for the unformatted drive via call to hd_bpb at: https://hg.pushbx.org/ecm/edrdos/file/tip/drbio/disk.asm#l2917 This default bpb is a FAT-16 bpb for in my case a 2GB volume, disrespecting the MBR partition type. This default bpb is registered unaltered in the unit descriptor table, because at line 2920ff. no jump instruction is found at the start of the boot sector, which by DRBIO is used as a test if a bpb exists the first partition sector.
Better link: https://hg.pushbx.org/ecm/edrdos/file/e1dbcad5136e/drbio/disk.asm#l2917 (This will link to the specific revision even if newer revisions change this file.)
Thanks! In the future I will link to specific revisions.
Regarding the issue, I think there is not much I can do from a FDISK perspective. Perhaps DRBIO can be altered to respect the partition type when creating a default BPB for unpartitioned disks. The variable parttype seems to be properly initialized from the partition table when hd_bpb runs. I think I can manage to do the required changes if you are interested but do not want to work on it yourself. This would make FreeDOS Format choose the right filesystem type when formatting under edrdos kernel, I hope.
I think I can manage to do the required changes if you are interested but do not want to work on it yourself.
Sure, I'd be happy to include your contribution.
Fine, then I will start working on it. May take some time. I have little to no experience with the EDR-DOS codebase yet...
@ecm-pushbx please see this hopefully minimally invasive patch to correct formatting issue with FreeDOS format under EDR-DOS. If you need this in some other form than a patch, drop me a note.
I left a few comments with what I think could be bugs in disk.asm. I did not audit the whole file, though...
I will look into this more at a later time, probably by the next weekend. I did glance at it. Here's a part of the comments you added:
BPB_SIZE et al. are not read from the BPB of the hard disk partition but always constructed from information stored in the partition table. This may impose a problem where these values disagree.
What do you mean by "et al"? I do agree that the sectors per image should be read from the (E)BPB, not the partition table.
What do you mean by "et al"? I do agree that the sectors per image should be read from the (E)BPB, not the partition table.
BPB_HIDDEN is also calculated from partition table instead taken from BPB. I tend to think that this is ok. But the actual size of the file system could be smaller than the partition. So BPB_SIZE should be taken from the BPB.
Yes, I agree the hidden sectors are fine to read from the partition table.
Took longer than I wanted to, but I added your contribution (unchanged) to my repo now: https://hg.pushbx.org/ecm/edrdos/rev/ed7b1285ad45
I studied your changes a bit and found that if I set a FAT16 file system type in the partition table, but with a small Sectors per Image of eg 2880, and an all-zeroed boot sector, then you can create a "FAT16" file system with too few clusters to actually be detected as FAT16. This does not cause any problems however. As I noted in a subsequent changeset: https://hg.pushbx.org/ecm/edrdos/rev/a074b5feec9c
; This branch can be taken even if 4 SpC (2 KiB clusters)
; would lead to misdetecting the FAT as FAT12. However,
; the only ill effect is that we may allocate too many
; FAT sectors for the assumed FAT16 in this case.
I also dropped an es: override. The immediately surrounding instructions also do not use it, although other accesses to the UPB do. During tracing the ds and es values both were the same (70h) however. https://hg.pushbx.org/ecm/edrdos/rev/a3d3198267bb
Finally I addressed your worries about the partition table entry size being preferred over the BPB indicated size. I changed it so the smaller of the two is used: https://hg.pushbx.org/ecm/edrdos/rev/8d3b6d9a415db + https://hg.pushbx.org/ecm/edrdos/rev/4d0f9156d536
By the way, studying the sources it seems that your fix was intended all along: The only branch to hd_bpb30 is for dx >= 256, but the code there checks for smaller dx values which was redundant before your patch. Another oversight of the enhancement project. https://hg.pushbx.org/ecm/edrdos/file/4d0f9156d536/drbio/disk.asm#l3149
Build will be in https://pushbx.org/ecm/download/edrdos.zip in about 30min.
@ecm-pushbx thanks for the detailed response. Always a pleasure to read your comments. Are you interested in further exchange of code / knowledge? I continued working on the code by implementing a replacement for the FIXUPP utility: https://github.com/SvarDOS/edrdos/blob/main/ltools/fixupp.c
I eventually want to replace most of the proprietary tools with open source ones, with COMPBIOS and COMPBDOS being the next ones.
I also made DRBIO.SYS and DRDOS.SYS linking with Watcom WLINK. DRBIO.SYS is an exact replica of the WarpLink one. DRDOS.SYS differs very slightly at the end of the file from the LINKCMD version, but it seems to be a functional binary. Details are here: https://github.com/SvarDOS/edrdos/issues/4
I further created Watcom makefiles for DRBIO and DRDOS, mainly to reduce the build time (it is a little bit slow under my DosBox).
I did not incorporate your latest changes yet, but plan to do so in the next days.
Greetings, Bernd
@ecm-pushbx thanks for the detailed response. Always a pleasure to read your comments.
You're making me blush!
Are you interested in further exchange of code / knowledge?
In principle yes, but I probably took care of the low-hanging fruit already. I'm probably not cut out to be a full-time maintainer of EDR-DOS, I just wanted to collect some of the patches that were floating about.
I'd still be interested in learning how to combine the BIO and the DOS file into a single kernel file to then combine this with my iniload and inicomp stages, changing the kernel to use the native lDOS / RxDOS.3 load protocol. This protocol is described some in the lDOS boot docs: https://pushbx.org/ecm/doc/ldosboot.htm#protocol-sector-iniload
One of the advantages to booting off of lDOS iniload is that it can be used as a number of different format kernels (unfortunately not as the current EDR-DOS DRBIO load protocol), quoth https://pushbx.org/ecm/doc/ldebug.htm#buildingprocess
The bootable executables can be used as MS-DOS 6 protocol IO.SYS, MS-DOS 7/8 IO.SYS, PC-DOS 6/7 IBMBIO.COM, FreeDOS KERNEL.SYS, RxDOS.3 RXDOS.COM, or as a Multiboot specification or Multiboot2 specification kernel. In any kernel load protocol case, the root FS that is being loaded from should be a valid FAT12, FAT16, or FAT32 file system on an unpartitioned (super)floppy diskette (unit number up to 127) or MBR-partitioned hard disk (unit number above 127). In addition, the bootable executables also are valid 86-DOS application programs that can be loaded in EXE mode either as application or as device driver. (Internally, all the .com files are MZ executables with a header, but they are named with a .COM file name extension for compatibility.)
I wrote about why to prefer a single-file kernel load in the forum, as well: https://www.bttr-software.de/forum/forum_entry.php?id=20762&page=1&category=0&order=time
I continued working on the code by implementing a replacement for the FIXUPP utility: https://github.com/SvarDOS/edrdos/blob/main/ltools/fixupp.c
I will have to try compiling this with tkchia's gcc-ia16.
I eventually want to replace most of the proprietary tools with open source ones, with COMPBIOS and COMPBDOS being the next ones.
Good ideas all around. I have considered (both for EDR-DOS's and MSDebug's closed-source-but-free tools) to perhaps reverse-engineer the tools, which is made easier in that assemblers and linkers are not sensitive to the timing of their performance. As opposed to PD ZModem: https://pushbx.org/ecm/dokuwiki/blog/pushbx/2022/1031_unmak_explanations_and_plans
I also made DRBIO.SYS and DRDOS.SYS linking with Watcom WLINK. DRBIO.SYS is an exact replica of the WarpLink one. DRDOS.SYS differs very slightly at the end of the file from the LINKCMD version, but it seems to be a functional binary. Details are here: SvarDOS/edrdos#4
I further created Watcom makefiles for DRBIO and DRDOS, mainly to reduce the build time (it is a little bit slow under my DosBox).
I don't have OpenWatcom installed usually on my development systems. WarpLink is a win for me because it is freer than OW, as yet: https://github.com/open-watcom/open-watcom-v2/discussions/271
I did not incorporate your latest changes yet, but plan to do so in the next days.
Glad to help!
Quoting the relevant bit from the unmak / PDZM post:
The easiest way to get these logs is to trace the execution of the original program using a command like
tp FFFFFF. However, it is difficult to obtain trace log code coverage for all the code of a nontrivial program. And another problem is that in the case of PDZM, tracing makes the application too slow to successfully transfer a file.
In principle yes, but I probably took care of the low-hanging fruit already. I'm probably not cut out to be a full-time maintainer of EDR-DOS, I just wanted to collect some of the patches that were floating about.
I can perfectly understand that ;-)
I'd still be interested in learning how to combine the BIO and the DOS file into a single kernel file to then combine this with my iniload and inicomp stages, changing the kernel to use the native lDOS / RxDOS.3 load protocol. This protocol is described some in the lDOS boot docs: https://pushbx.org/ecm/doc/ldosboot.htm#protocol-sector-iniload
Apart from compatibility it surely makes sense to combine both, from multiple perspectives: code removal, kernel size...!
One of the advantages to booting off of lDOS iniload is that it can be used as a number of different format kernels (unfortunately not as the current EDR-DOS DRBIO load protocol), quoth https://pushbx.org/ecm/doc/ldebug.htm#buildingprocess
I previously had a look at lDOS (the mbr code) when thinking about what to do with the current RANISH loaders, and if I should replace them. But then Willi Spiegl came along and I got stuck with FDISK. I do not think that I revisit RANISH in the next time. I prefer working on a software that is truely free. lDOS and a merged DRBIO/DRDOS kernel would be a good fit, I guess.
I continued working on the code by implementing a replacement for the FIXUPP utility: https://github.com/SvarDOS/edrdos/blob/main/ltools/fixupp.c
I will have to try compiling this with tkchia's gcc-ia16.
If it does not compile drop me a note. Then I will fix it. I have to touch the source anyway to cut off some rough edges, do some more input file validation...
I eventually want to replace most of the proprietary tools with open source ones, with COMPBIOS and COMPBDOS being the next ones.
Good ideas all around. I have considered (both for EDR-DOS's and MSDebug's closed-source-but-free tools) to perhaps reverse-engineer the tools, which is made easier in that assemblers and linkers are not sensitive to the timing of their performance. As opposed to PD ZModem: https://pushbx.org/ecm/dokuwiki/blog/pushbx/2022/1031_unmak_explanations_and_plans
Ouch. I once had a software bug (I think it was HTMLHELP) that only triggered if debug info was NOT enabled when compiling.
I don't have OpenWatcom installed usually on my development systems. WarpLink is a win for me because it is freer than OW, as yet: open-watcom/open-watcom-v2#271
Yes, WarpLink is really nice. But runs only under DOS (emulation) I guess?!? Both WarpLink and WLink are better than the proprietary LINKCMD :-) I will try to make DRDOS also link with WarpLink. That should not be so hard. Always good to have a choice ;-) In the current make.bat, DRDOS.SYS linking is still done with LINKCMD...
I do not think that I revisit RANISH in the next time. I prefer working on a software that is truely free. lDOS and a merged DRBIO/DRDOS kernel would be a good fit, I guess.
Isn't the version of ranish you've worked on also free software with sources?
I do not think that I revisit RANISH in the next time. I prefer working on a software that is truely free. lDOS and a merged DRBIO/DRDOS kernel would be a good fit, I guess.
Isn't the version of ranish you've worked on also free software with sources?
You are correct, I should have a break. I mixed it up with another software I worked on :-(
Anyway. I do not plan to make any major improvements to it in the next time. The partition manager part is LBA enabled and supports 2 TiB disks. LBA-enabling the advanced boot manager would be more time I want to invest. Perhaps anytime in the future...
This is now prevented by the EDR kernel via https://github.com/SvarDOS/edrdos/commit/4dbe13473d65b579e4c2cea4f5ca2016aed6567d, https://github.com/SvarDOS/edrdos/commit/a74e21180e272a127d5b37d8639a75a7de2a383d