skx
We're missing some functions for HiSoft C
There are two versions of the HiSoft C compiler here:
- https://github.com/davidly/cpm_compilers
- v135 - can be made to work with tweaks
- v3.09 - seems to hang
Anyway both require some missing syscalls to be implemented, and getting those added and tested properly, will close this bug.
We need to add, at least:
- PUNCH
- Add READER too for symmetry, I guess.
- F_TIMEDATE
More generally it would be nice if we could add support for arbitrary "fake" syscalls. Perhaps something like:
- If syscall exists use it.
- If os.Getenv("SYSCALL_124_FAIL" ) exists return a fail code
- If os.Getenv("SYSCALL_124_OKAY" ) exists return a pass code
- Otherwise abort with "unimplemented", as we do now.
Might be a bit messy, but it would save recompilation for users if there's just one syscall that is missing.
I'm going in circles over this.
I think I've got a couple of bugs, I think that "F_DELETE" doesn't close the file if it's empty which is causing failures. I think that F_SIZE is not doing the right thing if the file is open.
It's not as basic as a missing syscall though. Shame.
Roughly speaking the plan of attack is to run the v3.09 compiler under ntvcm from @davidly with tracing enabled, and compare the syscalls made.
~/ntvcm/ntvcm -t C309.COM SIEVE.C -O -LF -DHISOFTC
vs
cpmulator --input=stty --log-path my.log C309.COM SIEVE.C -O -LF -DHISOFTC
We'll see if that step by step comparison gets us further. At the end of the day I want this to work, but going round in circles any more will make me close this as "Won't fix" so easy steps :)
OK progress:
- Added BDOS function 102
- Added BIOS function 6.
Now looking at the tracing information both emulators do stuff with $$EXEC.$$$, and then move on to reading from the file $EXEC.COM my emulator aborts, or seems to diverge in behaviour from there. Interestingly my emulator logs:
{"time":"2025-05-17T14:36:12.854861691+03:00","level":"DEBUG","msg":"result:OK","function":"SysCallFileOpen","name":"$EXEC.COM","drive":"A","result":"$EXEC.COM","fcb":63912,"handle":6,"record_count":9,"file_size":1152}
{"time":"2025-05-17T14:36:12.854876034+03:00","level":"INFO","msg":"BDOS","name":"F_DMAOFF","type":"BDOS","syscall":26,"syscallHex":"0x1A","registers":{"AF":"80A8","BC":"F91A","DE":"0100","HL":"F923"}}
{"time":"2025-05-17T14:36:12.854885451+03:00","level":"INFO","msg":"BDOS","name":"F_READ","type":"BDOS","syscall":20,"syscallHex":"0x14","registers":{"AF":"00C0","BC":"0014","DE":"F9A8","HL":"0180"}}
{"time":"2025-05-17T14:36:12.854895569+03:00","level":"DEBUG","msg":"SysCallRead","dma":256,"fcb":63912,"handle":6,"offset":5242880}
NOTE: "offset":5242880 (!)
ntvcm logs:
bdos function 15: open file, bc 000f, de fea4, hl fece
FCB at address fea4:
drive: 0 == A
filename: '$EXEC '
filetype: 'COM'
R S A: 0 0 0
ex: 0
s1: 26
s2: 26
rc: 26
cr: 0
r0: 254
r1: 92
r2: 0
opening file '$EXEC.COM' for pfcb 0x50e504
could not find an open file entry for '$EXEC.COM'; that might be OK
file opened successfully, record count: 9
bdos function 26: set dma address, bc fe1a, de 0100, hl fe1f
updating DMA address; D 256 = 0x100
bdos function 20: read sequential, bc fe14, de fea4, hl 0180
FCB at address fea4:
drive: 0 == A
filename: '$EXEC '
filetype: 'COM'
R S A: 0 0 0
ex: 0
s1: 26
s2: 0
rc: 9
cr: 0
r0: 254
r1: 92
r2: 0
found file entry '$EXEC.COM'
file size: 0x480 = 1152, current 0 = 0, dma 0x100 = 256
0000 ed 7b 06 00 0e 19 cd 05 00 32 73 04 11 ff 00 0e : 20 cd 05 00 32 76 04 21 5c 00 11 cd 03 01 10 00 .{.......2s..... ...2v.!\.......
0020 ed b0 11 cd 03 0e 0f cd 05 00 3c 20 09 21 c5 03 : cd f8 02 c3 0a 02 2a 06 00 11 a2 04 37 ed 52 eb ..........< .!........*.....7.R.
0040 ed 4b a2 04 21 a4 04 79 b0 28 22 0b 7e 23 e5 66 : 6f e5 d5 11 7a 01 b7 ed 52 d1 e1 38 0c 7e 23 e5 .K..!..y.(".~#.fo...z...R..8.~#.
0060 66 6f 19 eb e3 72 2b 73 d1 e1 23 18 da 21 a1 04 : 19 eb 21 a1 04 01 26 03 ed b8 18 00 2a 06 00 22 fo...r+s..#..!....!...&.....*.."
new offset: 128, s2 0, ex 0, cr 1
bdos function 26: set dma address, bc 001a, de 0180, hl 0000
updating DMA address; D 384 = 0x180
bdos function 20: read sequential, bc 0014, de fea4, hl 0200
FCB at address fea4:
drive: 0 == A
filename: '$EXEC '
filetype: 'COM'
R S A: 0 0 0
ex: 0
s1: 26
s2: 0
rc: 9
cr: 1
r0: 254
r1: 92
r2: 0
found file entry '$EXEC.COM'
file size: 0x480 = 1152, current 0x80 = 128, dma 0x180 = 384
i.e. There you see that same open, and then two reads with a starting offset of 0x0000.
One thing about HiSoft C v3.09 is that the h register must be 0 on return from bdos calls. The compiler and the apps it generates look at hl for bdos return values, not just a or l. I'm just guessing.
I'm making some progress - I now reach a point where there is a single read of $$EXEC.$$$ which works, and then I get an error and a message "Fmt err" written to the console:
fileopen 973: $$EXEC.$$$ Record-Count: 6
read 63786: $$EXEC.$$$
Read 128 bytes
Updated sequential ptr to 128
CR is now 1/6
Return code 0
Fmt err
I think I'm accounting for the HL/B return code, and I've found a couple of areas in my code where I was making bad assumptions. For example if I closed a file with $ in the name I'd try to truncate - which is something that is necessary for the processing the files generated by SUBMIT.
I notice there's some weirdness with the last read in that the file open has a FCB at 973, but the next read is using a different FCB address. But that seems to be logged by your emulator too:
bdos function 15: open file, bc 000f, de 03cd, hl 006c
FCB at address 03cd:
drive: 0 == A
filename: '$$EXEC '
filetype: '$$$'
R S A: 0 0 0
ex: 0
s1: 0
s2: 0
rc: 0
cr: 0
r0: 0
r1: 0
r2: 0
opening file '$$EXEC.$$$' for pfcb 0x4fea2d
could not find an open file entry for '$$EXEC.$$$'; that might be OK
file opened successfully, record count: 6
bdos function 26: set dma address, bc 001a, de fe47, hl 0000
updating DMA address; D 65095 = 0xfe47
bdos function 20: read sequential, bc 0014, de fe26, hl 0000
FCB at address fe26:
drive: 0 == A
filename: '$$EXEC '
filetype: '$$$'
R S A: 0 0 0
ex: 0
s1: 0
s2: 0
rc: 6
cr: 0
r0: 0
r1: 0
r2: 0
found file entry '$$EXEC.$$$'
file size: 0x300 = 768, current 0 = 0, dma 0xfe47 = 65095
0000 84 00 81 6a 00 00 43 50 50 20 20 20 20 20 43 4f : 4d 00 00 00 00 00 2d 44 43 50 4d 20 20 20 43 20 ...j..CPP COM.....-DCPM C
0020 20 00 00 00 00 00 20 20 20 20 20 20 20 20 20 20 : 20 00 00 00 00 38 20 2d 44 43 50 4d 20 2d 44 48 ..... ....8 -DCPM -DH
0040 49 5f 54 45 43 48 5f 43 20 2d 44 7a 38 30 20 2d : 44 48 49 53 4f 46 54 43 20 2d 49 20 53 49 45 56 I_TECH_C -Dz80 -DHISOFTC -I SIEV
0060 45 2e 43 20 24 43 54 4d 50 31 2e 24 24 24 81 53 : 00 00 50 31 20 20 20 20 20 20 43 4f 4d 00 00 00 E.C $CTMP1.$$$.S..P1 COM...
new offset: 128, s2 0, ex 0, cr 1
bdos function 32: get/set user code, bc 0020, de fe00, hl 006c
bdos function 15: open file, bc 000f, de 005c, hl 0000
FCB at address 005c:
drive: 0 == A
filename: 'CPP '
filetype: 'COM'
R S A: 0 0 0
ex: 0
s1: 0
s2: 0
rc: 0
cr: 0
r0: 0
r1: 0
r2: 0
opening file 'CPP.COM' for pfcb 0x4fe6bc
So we both have the read of that file, but then you open CPP.COM and I don't. So something is weird there.
PR opened 👍
- More restrictive truncate-on-close behaviour for handling SUBMIT-created
$$$.SUBfiles.- In the past a single
$character in a filename triggered this.
- In the past a single
- Commit to 16-bit result in all BDOS (not BIOS) functions.
- Added a couple of missing syscalls (fake).
- Reset both the random and sequential offsets inside an FCB when a FileOpen or FileMake BDOS call is made.
- Minor logging tweaks.
The end result of that is a failure:
A>c309 sieve.c -O -LF -DHISOFTC
HI-TECH C COMPILER (CP/M-80) V3.09
Copyright (C) 1984-87 HI-TECH SOFTWARE
Fmt err
A>exit
"Fmt err" is text that comes from $EXEC.COM
So progress, but I'm not sure where to go next. I guess looking at the binary data that is read/written and seeing if that mismatches compared the working emulator.
I found a source for $EXEC.COM I need to work through it:
- https://github.com/agn453/HI-TECH-Z80-C/blob/master/cpm/EXEC.AS
In the meantime I've made progress in the open PR, and I guess I merge it since I see no regressions in either my test-suite, my random experimentation with BASIC, Zork, and Wordstar.
I just noticed I don't fully update the offsets in the read/write random functions - which is a bug, and I probably need to check the BDOS documentation to make sure I've not overlooked any other basic behaviour. Since "everything else" runs correctly my behaviour is probably pretty good, but if I've overlooked something obvious it might explain this failure. At least HiSoft v1.x works.
To recap the situation, from scratch:
- Clone this repository, and build the emulator.
-
go build . ; go install .
-
- Clone the repository that contains the compilers:
-
git clone [email protected]:davidly/cpm_compilers.git
-
- Run the emulator, and tell it what to run:
-
cpmulator -cd ~/cpm_compilers/hisoft\ hi-tech\ c\ v3.09/ -input stty -log-path foo -stuff "C309 SIEVE.C -O -LF -DHISOFTC\n" - Here the path is to v3.09 HiSoft C compiler.
- Expected a file named "SIEVE.COM" to be produced, rather than either an error or a hang.
-
I think it is now clear that the source of the issue is related to file I/O, and it might be as basic as not setting/resetting the Cr/S2/R0/R1/R2 fields in FCBs, etc, as David kindly commented in the PR referenced in this issue. However it might be more fundamentally related to either sequential or random I/O - frustratingly both are used!
My code:
OPEN_FILE(ENVIRON ) DE:4B88 CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
BDOS: A:FF B:00 H:00 L:FF
MAKE_FILE($$EXEC $$$) CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
OPEN_FILE($EXEC COM) DE:EAA4 CR:00 RC:1A EX:00 R0:FE R2:5C R2:00
RESULT CR:00 RC:09 EX:00 R0:FE R2:5C R2:00
BDOS: A:00 B:00 H:00 L:00
OPEN_FILE($$EXEC $$$) DE:03CD CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
RESULT CR:00 RC:06 EX:00 R0:00 R2:00 R2:00
BDOS: A:00 B:00 H:00 L:00
The working emulator:
OPEN_FILE(ENVIRON) DE:4B88 CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
BDOS: A:FF B:00 H:00 L:FF
MAKE_FILE($$EXEC.$$$) CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
OPEN_FILE($EXEC.COM) DE:FEA4 CR:00 RC:20 EX:00 R0:FE R1:5C R2:00
RESULT CR:00 RC:09 EX:00 R0:FE R2:5C R2:00
BDOS: A:00 B:00 H:00 L:00
OPEN_FILE($$EXEC.$$$) DE:03CD CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
RESULT CR:00 RC:05 EX:00 R0:00 R2:00 R2:00
BDOS: A:00 B:00 H:00 L:00
OPEN_FILE(CPP.COM) DE:005C CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
RESULT CR:00 RC:80 EX:00 R0:00 R2:00 R2:00
BDOS: A:00 B:00 H:00 L:00
Interesting differences - the DE register, i.e. FCB address, is very different. I see that incoming to the function the record count is different and different again once the function returns.
I guess that the FCB is pointing beyond the BDOS address, and that shouldn't matter. But why are the RC values wrong? Specifically I get RC as 6 for "$$EXEC.$$$":
frodo ~/cpm_compilers/hisoft hi-tech c v3.09 $ ls -l \$\$EXEC.\$\$\$
-rw-r--r-- 1 skx skx 768 May 31 10:40 '$$EXEC.$$$'
frodo ~/cpm_compilers/hisoft hi-tech c v3.09 $ expr 768 / 128
6
That seems fine, but the working emulator gets 5.
New plan; I'm obsessing over a lot of details and I suspect that the actual fault here is minimal - but its in a couple of places, so unless I get everything correct I'll still see failures.
So my plan is:
- Rewrite FileOpen, MakeFile, and File Close.
- Rewrite FileRead, FileWrite, and RandWrite/RandRead.
If I refer to the working project, and I check that my test cases pass and zork, etc, continue to run I hope I'll fix the real issue(s) by accident. If not I'll have satisfied myself that my I/O is not to blame and can move on.
I do think it is I/O, because I've made progress which has lead to opening "corrupt" filenames like "J CPP.COM", and having errors "Not found change disk and hit any key" or similar.
I made the basic changes to add missing syscalls. Unfortunately something I'm doing in File I/O is just off enough to break the 3.x version of the compiler.
I tried making incremental tweaks, but I can't quite tell what is wrong enough to get it right by accident. Rewriting all the file I/O primitives in https://github.com/skx/cpmulator/pull/240 still resulted in failure to run.
While I'd love to solve this I'm gonna ignore it unless / until I get another report of "something isn't quite right with file I/O". I didn't spend too long working on the disassembly and source-reading of the compiler - I guess that should have been another way to start, but right now:
- Closing.
- Still broken.
- Won't / Can't fix.