We're missing some functions for HiSoft C

[skx]

There are two versions of the HiSoft C compiler here:

• https://github.com/davidly/cpm_compilers
  • v135 - can be made to work with tweaks
  • v3.09 - seems to hang

Anyway both require some missing syscalls to be implemented, and getting those added and tested properly, will close this bug.

We need to add, at least:

• PUNCH
  • Add READER too for symmetry, I guess.
• F_TIMEDATE

More generally it would be nice if we could add support for arbitrary "fake" syscalls. Perhaps something like:

• If syscall exists use it.
• If os.Getenv("SYSCALL_124_FAIL" ) exists return a fail code
• If os.Getenv("SYSCALL_124_OKAY" ) exists return a pass code
• Otherwise abort with "unimplemented", as we do now.

Might be a bit messy, but it would save recompilation for users if there's just one syscall that is missing.

[skx]

I'm going in circles over this.

I think I've got a couple of bugs, I think that "F_DELETE" doesn't close the file if it's empty which is causing failures. I think that F_SIZE is not doing the right thing if the file is open.

It's not as basic as a missing syscall though. Shame.

[skx]

Roughly speaking the plan of attack is to run the v3.09 compiler under ntvcm from @davidly with tracing enabled, and compare the syscalls made.

~/ntvcm/ntvcm -t  C309.COM SIEVE.C -O -LF -DHISOFTC

vs

cpmulator --input=stty --log-path my.log  C309.COM SIEVE.C -O -LF -DHISOFTC 

We'll see if that step by step comparison gets us further. At the end of the day I want this to work, but going round in circles any more will make me close this as "Won't fix" so easy steps :)

[skx]

OK progress:

• Added BDOS function 102
• Added BIOS function 6.

Now looking at the tracing information both emulators do stuff with $$EXEC.$$$, and then move on to reading from the file $EXEC.COM my emulator aborts, or seems to diverge in behaviour from there. Interestingly my emulator logs:

{"time":"2025-05-17T14:36:12.854861691+03:00","level":"DEBUG","msg":"result:OK","function":"SysCallFileOpen","name":"$EXEC.COM","drive":"A","result":"$EXEC.COM","fcb":63912,"handle":6,"record_count":9,"file_size":1152}
{"time":"2025-05-17T14:36:12.854876034+03:00","level":"INFO","msg":"BDOS","name":"F_DMAOFF","type":"BDOS","syscall":26,"syscallHex":"0x1A","registers":{"AF":"80A8","BC":"F91A","DE":"0100","HL":"F923"}}
{"time":"2025-05-17T14:36:12.854885451+03:00","level":"INFO","msg":"BDOS","name":"F_READ","type":"BDOS","syscall":20,"syscallHex":"0x14","registers":{"AF":"00C0","BC":"0014","DE":"F9A8","HL":"0180"}}
{"time":"2025-05-17T14:36:12.854895569+03:00","level":"DEBUG","msg":"SysCallRead","dma":256,"fcb":63912,"handle":6,"offset":5242880}

NOTE: "offset":5242880 (!)

ntvcm logs:

bdos function 15: open file, bc 000f, de fea4, hl fece
  FCB at address fea4:
    drive:    0 == A
    filename: '$EXEC   '
    filetype: 'COM'
    R S A:    0 0 0
    ex:       0
    s1:       26
    s2:       26
    rc:       26
    cr:       0
    r0:       254
    r1:       92
    r2:       0
  opening file '$EXEC.COM' for pfcb 0x50e504
  could not find an open file entry for '$EXEC.COM'; that might be OK
  file opened successfully, record count: 9
bdos function 26: set dma address, bc fe1a, de 0100, hl fe1f
  updating DMA address; D 256 = 0x100
bdos function 20: read sequential, bc fe14, de fea4, hl 0180
  FCB at address fea4:
    drive:    0 == A
    filename: '$EXEC   '
    filetype: 'COM'
    R S A:    0 0 0
    ex:       0
    s1:       26
    s2:       0
    rc:       9
    cr:       0
    r0:       254
    r1:       92
    r2:       0
  found file entry '$EXEC.COM'
  file size: 0x480 = 1152, current 0 = 0, dma 0x100 = 256
  0000  ed 7b 06 00 0e 19 cd 05 00 32 73 04 11 ff 00 0e : 20 cd 05 00 32 76 04 21 5c 00 11 cd 03 01 10 00  .{.......2s..... ...2v.!\.......
  0020  ed b0 11 cd 03 0e 0f cd 05 00 3c 20 09 21 c5 03 : cd f8 02 c3 0a 02 2a 06 00 11 a2 04 37 ed 52 eb  ..........< .!........*.....7.R.
  0040  ed 4b a2 04 21 a4 04 79 b0 28 22 0b 7e 23 e5 66 : 6f e5 d5 11 7a 01 b7 ed 52 d1 e1 38 0c 7e 23 e5  .K..!..y.(".~#.fo...z...R..8.~#.
  0060  66 6f 19 eb e3 72 2b 73 d1 e1 23 18 da 21 a1 04 : 19 eb 21 a1 04 01 26 03 ed b8 18 00 2a 06 00 22  fo...r+s..#..!....!...&.....*.."
  new offset: 128, s2 0, ex 0, cr 1
bdos function 26: set dma address, bc 001a, de 0180, hl 0000
  updating DMA address; D 384 = 0x180
bdos function 20: read sequential, bc 0014, de fea4, hl 0200
  FCB at address fea4:
    drive:    0 == A
    filename: '$EXEC   '
    filetype: 'COM'
    R S A:    0 0 0
    ex:       0
    s1:       26
    s2:       0
    rc:       9
    cr:       1
    r0:       254
    r1:       92
    r2:       0
  found file entry '$EXEC.COM'
  file size: 0x480 = 1152, current 0x80 = 128, dma 0x180 = 384

i.e. There you see that same open, and then two reads with a starting offset of 0x0000.

[davidly]

One thing about HiSoft C v3.09 is that the h register must be 0 on return from bdos calls. The compiler and the apps it generates look at hl for bdos return values, not just a or l. I'm just guessing.

[skx]

I'm making some progress - I now reach a point where there is a single read of $$EXEC.$$$ which works, and then I get an error and a message "Fmt err" written to the console:

fileopen 973: $$EXEC.$$$ Record-Count: 6
read 63786: $$EXEC.$$$
	Read 128 bytes
	Updated sequential ptr to 128
	CR is now 1/6
	Return code 0
Fmt err

I think I'm accounting for the HL/B return code, and I've found a couple of areas in my code where I was making bad assumptions. For example if I closed a file with $ in the name I'd try to truncate - which is something that is necessary for the processing the files generated by SUBMIT.

I notice there's some weirdness with the last read in that the file open has a FCB at 973, but the next read is using a different FCB address. But that seems to be logged by your emulator too:

bdos function 15: open file, bc 000f, de 03cd, hl 006c
  FCB at address 03cd:
    drive:    0 == A
    filename: '$$EXEC  '
    filetype: '$$$'
    R S A:    0 0 0
    ex:       0
    s1:       0
    s2:       0
    rc:       0
    cr:       0
    r0:       0
    r1:       0
    r2:       0
  opening file '$$EXEC.$$$' for pfcb 0x4fea2d
  could not find an open file entry for '$$EXEC.$$$'; that might be OK
  file opened successfully, record count: 6
bdos function 26: set dma address, bc 001a, de fe47, hl 0000
  updating DMA address; D 65095 = 0xfe47
bdos function 20: read sequential, bc 0014, de fe26, hl 0000
  FCB at address fe26:
    drive:    0 == A
    filename: '$$EXEC  '
    filetype: '$$$'
    R S A:    0 0 0
    ex:       0
    s1:       0
    s2:       0
    rc:       6
    cr:       0
    r0:       0
    r1:       0
    r2:       0
  found file entry '$$EXEC.$$$'
  file size: 0x300 = 768, current 0 = 0, dma 0xfe47 = 65095
  0000  84 00 81 6a 00 00 43 50 50 20 20 20 20 20 43 4f : 4d 00 00 00 00 00 2d 44 43 50 4d 20 20 20 43 20  ...j..CPP     COM.....-DCPM   C 
  0020  20 00 00 00 00 00 20 20 20 20 20 20 20 20 20 20 : 20 00 00 00 00 38 20 2d 44 43 50 4d 20 2d 44 48   .....           ....8 -DCPM -DH
  0040  49 5f 54 45 43 48 5f 43 20 2d 44 7a 38 30 20 2d : 44 48 49 53 4f 46 54 43 20 2d 49 20 53 49 45 56  I_TECH_C -Dz80 -DHISOFTC -I SIEV
  0060  45 2e 43 20 24 43 54 4d 50 31 2e 24 24 24 81 53 : 00 00 50 31 20 20 20 20 20 20 43 4f 4d 00 00 00  E.C $CTMP1.$$$.S..P1      COM...
  new offset: 128, s2 0, ex 0, cr 1
bdos function 32: get/set user code, bc 0020, de fe00, hl 006c
bdos function 15: open file, bc 000f, de 005c, hl 0000
  FCB at address 005c:
    drive:    0 == A
    filename: 'CPP     '
    filetype: 'COM'
    R S A:    0 0 0
    ex:       0
    s1:       0
    s2:       0
    rc:       0
    cr:       0
    r0:       0
    r1:       0
    r2:       0
  opening file 'CPP.COM' for pfcb 0x4fe6bc

So we both have the read of that file, but then you open CPP.COM and I don't. So something is weird there.

[skx]

PR opened 👍

• More restrictive truncate-on-close behaviour for handling SUBMIT-created $$$.SUB files.
  • In the past a single $ character in a filename triggered this.
• Commit to 16-bit result in all BDOS (not BIOS) functions.
• Added a couple of missing syscalls (fake).
• Reset both the random and sequential offsets inside an FCB when a FileOpen or FileMake BDOS call is made.
• Minor logging tweaks.

The end result of that is a failure:

A>c309 sieve.c -O -LF -DHISOFTC
HI-TECH C COMPILER (CP/M-80) V3.09
Copyright (C) 1984-87 HI-TECH SOFTWARE
Fmt err

A>exit

"Fmt err" is text that comes from $EXEC.COM

So progress, but I'm not sure where to go next. I guess looking at the binary data that is read/written and seeing if that mismatches compared the working emulator.

[skx]

I found a source for $EXEC.COM I need to work through it:

• https://github.com/agn453/HI-TECH-Z80-C/blob/master/cpm/EXEC.AS

In the meantime I've made progress in the open PR, and I guess I merge it since I see no regressions in either my test-suite, my random experimentation with BASIC, Zork, and Wordstar.

I just noticed I don't fully update the offsets in the read/write random functions - which is a bug, and I probably need to check the BDOS documentation to make sure I've not overlooked any other basic behaviour. Since "everything else" runs correctly my behaviour is probably pretty good, but if I've overlooked something obvious it might explain this failure. At least HiSoft v1.x works.

[skx]

To recap the situation, from scratch:

• Clone this repository, and build the emulator.
  • go build . ; go install .
• Clone the repository that contains the compilers:
  • git clone git@github.com:davidly/cpm_compilers.git
• Run the emulator, and tell it what to run:
  • cpmulator -cd ~/cpm_compilers/hisoft\ hi-tech\ c\ v3.09/ -input stty -log-path foo -stuff "C309 SIEVE.C -O -LF -DHISOFTC\n"
  • Here the path is to v3.09 HiSoft C compiler.
  • Expected a file named "SIEVE.COM" to be produced, rather than either an error or a hang.

I think it is now clear that the source of the issue is related to file I/O, and it might be as basic as not setting/resetting the Cr/S2/R0/R1/R2 fields in FCBs, etc, as David kindly commented in the PR referenced in this issue. However it might be more fundamentally related to either sequential or random I/O - frustratingly both are used!

[skx]

My code:

OPEN_FILE(ENVIRON    ) DE:4B88 CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
	BDOS: A:FF B:00 H:00 L:FF
MAKE_FILE($$EXEC  $$$) CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
OPEN_FILE($EXEC   COM) DE:EAA4 CR:00 RC:1A EX:00 R0:FE R2:5C R2:00
	RESULT CR:00 RC:09 EX:00 R0:FE R2:5C R2:00
	BDOS: A:00 B:00 H:00 L:00
OPEN_FILE($$EXEC  $$$) DE:03CD CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
	RESULT CR:00 RC:06 EX:00 R0:00 R2:00 R2:00
	BDOS: A:00 B:00 H:00 L:00

The working emulator:

OPEN_FILE(ENVIRON) DE:4B88 CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
	BDOS: A:FF B:00 H:00 L:FF
MAKE_FILE($$EXEC.$$$) CR:00 RC:00 EX:00 R0:00 R2:00 R2:00
OPEN_FILE($EXEC.COM) DE:FEA4 CR:00 RC:20 EX:00 R0:FE R1:5C R2:00
	RESULT CR:00 RC:09 EX:00 R0:FE R2:5C R2:00
	BDOS: A:00 B:00 H:00 L:00
OPEN_FILE($$EXEC.$$$) DE:03CD CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
	RESULT CR:00 RC:05 EX:00 R0:00 R2:00 R2:00
	BDOS: A:00 B:00 H:00 L:00
OPEN_FILE(CPP.COM) DE:005C CR:00 RC:00 EX:00 R0:00 R1:00 R2:00
	RESULT CR:00 RC:80 EX:00 R0:00 R2:00 R2:00
	BDOS: A:00 B:00 H:00 L:00

Interesting differences - the DE register, i.e. FCB address, is very different. I see that incoming to the function the record count is different and different again once the function returns.

I guess that the FCB is pointing beyond the BDOS address, and that shouldn't matter. But why are the RC values wrong? Specifically I get RC as 6 for "$$EXEC.$$$":

frodo ~/cpm_compilers/hisoft hi-tech c v3.09 $ ls -l \$\$EXEC.\$\$\$ 
-rw-r--r-- 1 skx skx 768 May 31 10:40 '$$EXEC.$$$'
frodo ~/cpm_compilers/hisoft hi-tech c v3.09 $ expr 768 / 128
6

That seems fine, but the working emulator gets 5.

[skx]

New plan; I'm obsessing over a lot of details and I suspect that the actual fault here is minimal - but its in a couple of places, so unless I get everything correct I'll still see failures.

So my plan is:

• Rewrite FileOpen, MakeFile, and File Close.
• Rewrite FileRead, FileWrite, and RandWrite/RandRead.

If I refer to the working project, and I check that my test cases pass and zork, etc, continue to run I hope I'll fix the real issue(s) by accident. If not I'll have satisfied myself that my I/O is not to blame and can move on.

I do think it is I/O, because I've made progress which has lead to opening "corrupt" filenames like "J CPP.COM", and having errors "Not found change disk and hit any key" or similar.