Better support for DOS 3.3 random-access text files

[fadden]

[ imported from Anonymous bug, https://sourceforge.net/p/ciderpress/bugs/1/ ]

Grab http://www.apple2.org.za/mirrors/ground.icaen.uiowa.edu/apple8/Games/FarFuture/TRADER.SDK

Convert to a disk image (e.g. DSK). Open up in AppleWin (etc). Look at the random access text file "SHIPS", e.g. by running:

10 PRINT CHR$(4)"OPEN SHIPS,L100" 20 PRINT CHR$(4)"READ SHIPS,R0" 30 INPUT N 40 FOR I = 1 TO N 50 PRINT CHR$(4)"READ SHIPS,R";I 60 INPUT A$ 70 PRINT I;" > ";MID$(A$,52,33) 80 NEXT 90 PRINT CHR$(4)"CLOSE SHIPS"

Note that it has 27 records total (record 0 being a numeric count of the number of remaining records). 25 is "BLACK ORCHID SCOUT/COURIER". 26 is "X-51215 EXPRESS BOAT"

Now open up the SDK or DSK in CiderPress, and look at the SHIPS file, either as Converted Text or Hex Dump. Note that the BLACK ORCHID is the last record!

I believe CiderPress isn't handling some subtlety of random access text files. I tried "copying" the file within the disk image by extending the applesoft program to write out an identical file - same behavior. I then made the program output duplicate copies of the last line and was then able to view it in CiderPress.

[fadden]

Using the "open file" feature of the CiderPress disk sector viewer, the last two sectors of the file are:

00: b0 b0 a0 b1 b1 b1 a0 b1 b3 a0 b0 b9 a0 b0 b2 b1  00 111 13 09 021
10: b0 a0 b0 b0 b5 b0 a0 d3 a0 b7 a0 b4 a0 b0 b2 b0  0 0050 S 7 4 020
20: b0 b0 b0 a0 b1 b0 b1 ae b0 b3 b0 a0 b0 b5 a0 cd  000 101.030 05 M
30: c1 d2 c3 c8 a0 c8 c1 d2 d2 c9 c5 d2 a0 a0 a0 a0  ARCH HARRIER    
40: c6 c1 d4 a0 d4 d2 c1 c4 c5 d2 a0 a0 a0 a0 a0 a0  FAT TRADER      
50: a0 c9 cd a0 d3 a0 a0 a0 8d 8d 00 00 00 00 00 00   IM S   ........
60: d3 a0 b0 b1 b0 b0 a0 b2 b2 b2 a0 b0 b4 a0 b0 b0  S 0100 222 04 00
70: a0 b0 b0 b0 b3 a0 b0 b0 b4 b0 a0 d3 a0 b5 a0 b2   0003 0040 S 5 2
80: a0 b0 b0 b7 b5 b0 b0 a0 b0 b2 b9 ae b4 b3 b0 a0   007500 029.430 
90: b0 b1 a0 c4 c1 c6 c6 cf c4 c9 cc a0 a0 a0 a0 a0  01 DAFFODIL     
a0: a0 a0 a0 a0 d3 c3 cf d5 d4 af c3 cf d5 d2 c9 c5      SCOUT/COURIE
b0: d2 a0 a0 a0 a0 c9 cd a0 d0 a0 a0 a0 8d 8d 00 00  R    IM P   ....
c0: 00 00 00 00 d3 a0 b0 b1 b0 b0 a0 b2 b2 b2 a0 b0  ....S 0100 222 0
d0: b4 a0 b0 b0 a0 b0 b0 b0 b3 a0 b0 b0 b4 b0 a0 d3  4 00 0003 0040 S
e0: a0 b6 a0 b2 a0 b0 b0 b7 b5 b0 b0 a0 b0 b2 b9 ae   6 2 007500 029.
f0: b4 b3 b0 a0 b0 b1 a0 c2 cc c1 c3 cb a0 cf d2 c3  430 01 BLACK ORC

00: c8 c9 c4 a0 a0 a0 a0 a0 d3 c3 cf d5 d4 af c3 cf  HID     SCOUT/CO
10: d5 d2 c9 c5 d2 a0 a0 a0 a0 c9 cd a0 d0 a0 a0 a0  URIER    IM P   
20: 8d 8d 00 00 00 00 00 00 d8 a0 b0 b1 b0 b0 a0 b4  ........X 0100 4
30: b0 b4 a0 b0 b2 a0 b0 b0 a0 b0 b0 b0 b1 a0 b0 b0  04 02 00 0001 00
40: b4 b0 a0 d5 a0 b9 a0 b2 a0 b0 b0 b7 b5 b0 b0 a0  40 U 9 2 007500 
50: b0 b7 b0 ae b6 b5 a0 a0 b0 b1 a0 d8 ad b5 b1 b2  070.65  01 X-512
60: b1 b6 a0 a0 a0 a0 a0 a0 a0 a0 a0 a0 c5 d8 d0 d2  16          EXPR
70: c5 d3 d3 a0 c2 cf c1 d4 a0 a0 a0 a0 a0 c9 cd a0  ESS BOAT     IM 
80: d0 a0 a0 a0 8d 8d 00 00 00 00 00 00 00 00 00 00  P   ............
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

The file viewer shows:

000009d0: b4 a0 b0 b0 a0 b0 b0 b0 b3 a0 b0 b0 b4 b0 a0 d3  4 00 0003 0040 S
000009e0: a0 b6 a0 b2 a0 b0 b0 b7 b5 b0 b0 a0 b0 b2 b9 ae   6 2 007500 029.
000009f0: b4 b3 b0 a0 b0 b1 a0 c2 cc c1 c3 cb a0 cf d2 c3  430 01 BLACK ORC
00000a00: c8 c9 c4 a0 a0 a0 a0 a0 d3 c3 cf d5 d4 af c3 cf  HID     SCOUT/CO
00000a10: d5 d2 c9 c5 d2 a0 a0 a0 a0 c9 cd a0 d0 a0 a0 a0  URIER    IM P   
00000a20: 8d 8d                                            ..              

The code is (sadly) working as intended. The problem is that, if this were a sequential-access text file, we'd want to stop at the first 00 byte in the last sector. IIRC, this is because sequential-access text files can leave junk bytes at the end if they get rewritten.

This is wrong for random-access text files, as noted in the comment in diskimg/DOS33.cpp:

* For Text, there are two situations: sequential and random.  For
* sequential text files, we just need to find the first 00 byte.  For
* random, there can be 00s everywhere, and in fact there can be holes
* in the T/S list.  The plan: since DOS doesn't let you "truncate" a
* text file, just scan the last sector for 00.  The length is the
* number of previous T/S entries * 256 plus the sector offset.
* --> This does the wrong thing for random-access text files, which
* need to retain their full length, and doesn't work right for sequential
* text files that (somehow) had their last block over-allocated.  It does
* the right thing most of the time, but we either need to be more clever
* here or provide a way to override the default (bool fTrimTextFiles?).

It might be possible to perform a more sophisticated scan that attempts to identify random access text file by looking for occurrences of 00 bytes outside the last sector. This won't work quite right on very short files, but it'd work a bit better than what we have now.

[fadden]

This will not be addressed in CiderPress. However, https://github.com/fadden/ciderpress2 has random-access text file support. It turns the text files into CSV.

For the "SHIPS" file, set the length to 100:

26
"A 0200 111 10 20 0082 0030 S 7 4 015000 037.080 04 BEOWULF          FREE TRADER      IM 240 "
"A 0200 111 10 20 0082 0030 S 7 4 015000 037.080 04 FANCY FREE       FREE TRADER      DA 240 "
[...]
"S 0100 222 04 00 0003 0040 S 5 2 007500 029.430 01 DAFFODIL         SCOUT/COURIER    IM P   "
"S 0100 222 04 00 0003 0040 S 6 2 007500 029.430 01 BLACK ORCHID     SCOUT/COURIER    IM P   "
"X 0100 404 02 00 0001 0040 U 9 2 007500 070.65  01 X-51216          EXPRESS BOAT     IM P   "