fluxengine
fluxengine copied to clipboard
davidgiven
AGAT 840K format support request
Dear David, Thank you very much for your project and support. Does it possible to add support of AGAT 840K format?
Agat is a very common soviet computer, based on Apple II. It use to formats: Apple II 140K and AGAT840K
Description of format - https://github.com/sintech/AGAT/blob/master/docs/agat-840k-format.txt files - http://oldpc.su/0/106.rar 01.flux and 01.raw - 1st disk (flux and raw data format) 02.flux and 02.raw - 2nd disk 10.flux and 11.flux - two other disks (read by latest client and fw of FluxEngine).
Thank you very much in advance!!!!
Thank you --- that's exactly the information I need! I'll get onto it when I get the chance.
Hello --- I've given a stab at this. The decoder sort of works, but it produces very bad results on the disk images you sent me. Can you confirm that the disk images don't contain any bad sectors? Do you have an AGAT?
Also, do you have any more information on the desync sequences? The disk images don't seem to match the text file. I did find this thread: https://torlus.com/floppy/forum/viewtopic.php?t=1385 and it has multiple descriptions, which don't match.
See #457 for the code, if you're interested.
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
0. 0 ..C.....C...................C..........................B.......C...BBBB..BBBBBBB
0. 1 ...B.................C.............X........B.C?.............C.....BBBB..BBBBBB.
0. 2 ...BB..........................................X..B?..CB.......C...BB.B..BBBBBBB
0. 3 .C.B.............C..................C........C.....XC..B....C......B?BB..BBBBBBB
0. 4 ...B....................................?...?.......?..B?..........BBBB..BBBBB.B
0. 5 ...B...C................................X...X......CX..BX..........BBBB..BBBBBBB
0. 6 C..B.B..........C.........C........C......X..........C.B...........?.BB..XBBBBXB
0. 7 ...B........................................C..........B...C.......BBBB..BBBBBBB
0. 8 ...B.....................................C.............B.?.........BBB...BBBB.BB
0. 9 ...B.....................C.......B...B.........X..C....B.X.....B....BBB.CBBBBBBB
0.10 ...B...........C..................C........C...........B..?........BBB....BBBBB?
0.11 ...B.C.........................C........C..............B..X........BBBB...B?BBBX
0.12 ...B.B..................C.......?................C..X..B...........BBB....BB.BBB
0.13 ...BB...B.....C..................C..............?......B..........CBBB..C.BBBBBB
0.14 ...B...................?......C..........XC.....X......B.C..........BB....BBBBBB
0.15 ...B...................X.............?..........C....?.B.......?....BB....B.BBBB
0.16 BBBBB.....C............C.............X...............X..B......XB...BB.C..BBBBBB
0.17 BBBB.........C.....C............C.....C......B..........C...........BB....BBBBBB
0.18 BBB..B...C...................C......................................BB....BBBBBB
0.19 BBBB........C.....B...C..........B.............CX.............C.....BB.....BBBBB
0.20 BBBB............B.B.........................X.......?..C........C...BBC...BBBBBB
1. 0 .................B............................C...X................BBB....BBBBB.
1. 1 ....B......C.........C...........?..........B.........B......C.....BBB...BBBBBBB
1. 2 .................C...?...........X..C..........................C.X.BBB...B?BBBBB
1. 3 .C...................X.....C.................C........B............BBB...BXBBBBB
1. 4 ....B.......B............................B.........................BBB...BBBBB.B
1. 5 ............B......................................C..............BB.B...BBBBBBB
1. 6 C...........B...CB........C........C........C..............C.......BBB...XBBBBBB
1. 7 ....B.......B......................................................BBB..?BBBBBBB
1. 8 ......C.....B............................C........C?..............XBBB..XBBBBBBB
1. 9 ....B.......B..C.........C........C................XC...............BB...BBBBBBB
1.10 ............B..............................C..............C........BBB...BBBBBBB
1.11 ............B...........................C...B..?...................BBB...BBB.BBB
1.12 ............B...........C......................X.CX...............CBBB..CBBBBBBB
1.13 ............BBC.....C............C.....C.................C.B.......BBB...BBBBBBB
1.14 ....C.........................C....................................BBB...BBBBBBB
1.15 .......................C........................C.B................BBB...BB.BBBB
1.16 B.B..........C..................C................................C.BBB.C.BBB?BBB
1.17 BBB......C...................C........C.................C..........BBB...BBBXBBB
1.18 BBB.B..........................................C.......BX..........BBB...B.BBBBB
1.19 BBB.....B.............C...........................B....B......C....BBBC..BBBBBBB
1.20 BBB.........C.....C...X........C.....C.............................BBB...BBBBBBB
Dear David, Thank you very much for your support. And sorry for delay with answer (covid took 2,5 weeks if my life)
As I can see from sources (may be I can't read it correctly) you don't check the desync and trying to find the patterns (0xff956a and 0xff6a95) only. I think this is the root cause for so many "multiple conflicting copies of sector nn seen; ... Failed to read sector nn (conflicting data)" errors. Agat has software based FDC (on two 8255; one 8255 per disk side) and can use many different desync, but, as I know, only one usually used. Any desync has syncrobit error (missing clock '1').
I have tried read the one more disk and .img work with agat emulator. http://oldpc.su/0/201.rar
I hope you're better now!
Re the desync: do you have any more information on the exact bit layout? The documentation doesn't mention the missing clock bit. In fact, it says 'Desync: 0xA4, 2 ms zero level interval, 0xFF' which I can't really make sense of.
Sorry for delay with answer,
From real disk:
0xA4 with wrong sync bit (marked as V; last zero bit in 0x8924).
One bit skipped (?) after desync and 0xFF9569 (address field prologue)
Please see attached files.
agat_______.txt
if I correctly understand:
IBM PC desync 0xA1: desync made by changing one of sync bit. FDC will write totally 16 bits (8 bits data and 8 bits sync).
Agat 840k desync: desync made by adding one 0. FDC will write 0xA4 and one additional 0 (totally 17 bits; 1000 1001 0010 0100 0)
Thank you for that --- I've managed to improve the decode a fair bit. See the log below.
However, I don't think your description of the desync's quite right. The extra 0 marked in your diagram is actually the data bit of the next byte. I think the sync pattern is actually 0x12, and the 0xA4 is describing the clock bits, rather than the data. The bit pattern I'm actually seing is this:
* data: - - - X - - X - = 12
* flux: 10 10 10 01 00 10 01 00 = a924 (normal MFM encoding)
* magic: 10 00 10 01 00 10 01 00 = 8924 (with the magic clock bit removed)
This ends up being much more like a normal disk format.
Unfortunately, the reads are still full of bad sectors, which I'm continuing to investigate. Do you know if the disks are actually good --- in other words, if the bad sectors are real? (Currently I'm assuming not.)
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
0. 0 ................................C......................B...........BBBB..BBBBBBB
0. 1 ...B...C....................................B......................BBBB..BBBBBBB
0. 2 ...BB.............................................B....B...........BBBB..BBBBBBB
0. 3 ...B...................................................B...........BBBB..BBBBBBB
0. 4 ...B...................................................B...........BBBB..BBBBBBB
0. 5 ...B...................................................B...........BBBB..BBBBBBB
0. 6 ...B.B.................................................B...........BBBB..BBBBBBB
0. 7 ...B...................................................B...........BBBB..BBBBBBB
0. 8 ...B...................................................B...........BBB...BBBBBBB
0. 9 ...B.............................B...B.................B.......B...BBBB..BBBBBBB
0.10 ...B...................................................B...........BBB...BBBBBBB
0.11 ...B...................................................B...........BBBB...BBBBBB
0.12 ...B.B.................................................B...........BBB....BBBBBB
0.13 ...BB...B..............................................B...........BBB....BBBBBB
0.14 ...B...................................................B............BB....BBBBBB
0.15 ...B...................................................B............BB....BBBBBB
0.16 BBBBB...................................................B.......B...BB....BBBBBB
0.17 BBBB.........................................B......................BB....BBBBBB
0.18 BBBB.B..............................................................BB....BBBBBB
0.19 BBBB..............B..............B..................................BB....BBBBBB
0.20 BBBB............B.B.................................................BB....BBBBBB
1. 0 .................B.................................................BBB...BBBBBBB
1. 1 ....B.......................................B.........B............BBB...BBBBBBB
1. 2 ......................................................B............BBB...BBBBBBB
1. 3 ......................................................B............BBB...BBBBBBB
1. 4 ....B.......B............................B.........................BBB...BBBBBBB
1. 5 ............B.....................................................BBBB...BBBBBBB
1. 6 ............B....B.................................................BBB...BBBBBBB
1. 7 ....B.......B......................................................BBB...BBBBBBB
1. 8 ............B......................................................BBB...BBBBBBB
1. 9 ....B.......B......................................................BBB...BBBBBBB
1.10 ............B..............C.......................................BBB...BBBBBBB
1.11 ............B...............................B......................BBB...BBBBBBB
1.12 ............B......................................................BBB...BBBBBBB
1.13 ............BB.............................................B.......BBB...BBBBBBB
1.14 ...................................................................BBB...BBBBBBB
1.15 ..................................................B................BBB...BBBBBBB
1.16 B.B................................................................BBB...BBBBBBB
1.17 BBB................................................................BBB...BBBBBBB
1.18 BBB.B..................................................B...........BBB...BBBBBBB
1.19 BBB.....B.........................................B....B...........BBB...BBBBBBB
1.20 BBB...........................C........................B...........BBB...BBBBBBB
Good sectors: 2828/3360 (84%)
Missing sectors: 0/3360 (0%)
Bad sectors: 532/3360 (15%)
The extra 0 marked in your diagram is actually the data bit of the next byte. next byte is 0xff data 1111 1111 birstream 10101010 10101010 first databit is 1, not 0. Informaton about extra 0 based on analyse of hardware of FDC.
also:
2.1 Desync: 0xA4, 2 ms zero level interval, 0xFF "2 ms zero level interval" - it is extra 0
All fixed, and we now get perfect reads! You might want to look at https://github.com/davidgiven/fluxengine/pull/457/commits/39d6b0525f7eeac37cfd04db1c4be3d7a8ea9e1b if you want a laugh...
Do you actually have one of these machines? If so, could you format a blank disk, image it, and file a PR on https://github.com/davidgiven/fluxengine-testdata, please?
Regarding the desync sequence: I've figured it out --- the sequence on disk is 0x12 with a missing clock bit. Bit this is generated by emitting a standard MFM 0xA4 with an extra bit after it. That way, the MFM encoder hardware doesn't have to be able to generate sequences with non-standard clock bits, which is very clever!
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
0. 0 ................................................................................
0. 1 ................................................................................
0. 2 ................................................................................
0. 3 ................................................................................
0. 4 ................................................................................
0. 5 ................................................................................
0. 6 ................................................................................
0. 7 ................................................................................
0. 8 ................................................................................
0. 9 ................................................................................
0.10 ................................................................................
0.11 ................................................................................
0.12 ................................................................................
0.13 ................................................................................
0.14 ................................................................................
0.15 ................................................................................
0.16 ................................................................................
0.17 ................................................................................
0.18 ................................................................................
0.19 ................................................................................
0.20 ................................................................................
1. 0 ................................................................................
1. 1 ................................................................................
1. 2 ................................................................................
1. 3 ................................................................................
1. 4 ................................................................................
1. 5 ................................................................................
1. 6 ................................................................................
1. 7 ................................................................................
1. 8 ................................................................................
1. 9 ................................................................................
1.10 ................................................................................
1.11 ................................................................................
1.12 ................................................................................
1.13 ................................................................................
1.14 ................................................................................
1.15 ................................................................................
1.16 ................................................................................
1.17 ................................................................................
1.18 ................................................................................
1.19 ................................................................................
1.20 ................................................................................
Good sectors: 3360/3360 (100%)
Missing sectors: 0/3360 (0%)
Thank you very much!!!!
Do you actually have one of these machines? Yes, I have, but on stock and I need time to get it working. I will try make blank formatted disk.
I will check #457 with few disks Monday. When you will add Agat 840K to main FluxEngine release?
Now it's reading images, I can do so more or less immediately --- I'm just doing some basic documentation now. If you have any interesting links I'd be happy to add them. It'd also be good to add write support, but these require me (or someone!) to have access to an actual machine for testing, or else there's little point.
I can't check it:
-
latest release don't write anything in CLI D:\fluxengine>fluxengine.exe D:\fluxengine>fluxengine.exe read --help
-
GUI answer "This format is read-only" after press the "Read image" button
-
if I trying to compile #457 I have got errors: make: wx-config: No such file or directory make: wx-config: No such file or directory In file included from src/gui/main.cc:2: src/gui/gui.h:4:10: fatal error: wx/wx.h: No such file or directory
Also I have got the problem with reading disk.
# ./fluxengine read agat840 --flux_source.drive.revolutions 3 -o 1.dsk Using FluxEngine 7403020917073374 Measuring rotational speed... 199ms 0.0: 600 ms in 163776 bytes 125 records, 62 sectors; 1.94us clock (515kHz); 21 distinct sectors; logical track 0.0; 15872 bytes decoded. 0.1: 600 ms in 165824 bytes 121 records, 62 sectors; 1.97us clock (508kHz); 21 distinct sectors; Failed to read sector 11 (present but no data found); 5 retries remaining ..... 0.1: 600 ms in 165696 bytes 121 records, 62 sectors; 1.96us clock (511kHz); 21 distinct sectors; Failed to read sector 11 (present but no data found); giving up logical track 0.1; 90624 bytes decoded.
but I can see the header:
and data field
The flux is here oldpc.su/0/ff.flux
I think you need mark data field without trailing 0x5A as BAD_CHECKSUM, not DATA_MISSING. Or it is incorrect?
Re the CLI: that's a bug. I've fixed it and it's rebuilding now. Sorry! #501
Re the GUI: yes, that's intentional --- to read disks you need to press 'read flux' and then 'write image'. To write disks you do 'read image' / 'write flux'.
Re building: there are some new dependencies. See the 'building the client' section in https://cowlark.com/fluxengine/doc/building.html.
Re DATA_MISSING: yes, that's a good idea.
Re the missing sector: I can see the same thing. It looks like that header isn't been detected, so the record which sector 11 is in contains the sector 12 data too. Here's the middle of the record:
001f0 : 11 45 11 29 29 12 49 25 55 54 a5 2a 51 45 55 45 : .E.)).I%UT.*QEUE
00200 : 54 55 11 12 91 24 95 2a 45 11 11 11 89 24 55 55 : TU...$.*E....$UU
00210 : 49 11 14 44 89 24 55 55 14 44 49 11 2a 95 12 4a : I..D.$UU.DI.*..J
00220 : 4a 44 92 44 92 44 92 44 92 4a 49 4a a4 aa 54 91 : JD.D.D.D.JIJ..T.
You can see the 89 24 55 55 appearing twice, once for the header and once for the data. However, there's no actual header data between them. And the preceding byte (11) is invalid.
What program are you using to see the MFM decoded data?
to read disks you need to press 'read flux' and then 'write image'
It's mean that the retry to read the physical disk impossible ? Just one or few revolutions without checking format and checksums?
I will check the sector 11 tomorrow.
What program are you using to see the MFM decoded data?
Oh. I have added -V option to the old version of fe-inspect (I have sent it to you few years ago). This viewer can shift data by 0-15 bits (in cycle: shift left to 0,1,2,3,.....15,0,1,2...) and show sync bits errors by 'bold' (switchable FM/MFM mode). It is very nice for detect the disk for which computer and it is good tool for unknown formats. I need to move it to new version of FluxEngine, but i haven't time :-(
I have checked the sector 11 again and can't find anything wrong.
You can see end of previous sector (0x5a), GAP (22 pcs 0xAA) and 0xA4 desync:
(shifted to left by 1 bit) You can see correct sector header (marked red) and 0xA4 desync as start of data field (marked green)
(shifted to left by 1 bit) data field prologue (marked green) and data field epilogue (marked blue). No any desync (sync error bit) in sector data.
That is wrong?
and one more question: Why you ignore the missing sector? I think the missing sector is a error and we need retry to read disk.
As I know Agat don't use index signal. And we can't determine the start of the track. And we got the next error:
- we have found the correct sector header
- we have found the correct data field prologue and start reading data
- but after (for example) 100 bytes we have reached the start of track.
- we can't find checksum and data field epilogue and marked the sector as B. But we lost the start of track and correct sector header.
I think that if checksum is wrong - we need "rewind" data field to start and try to find desync and sector header.
I can made flux for example if you need.
Hi!
Is there any chance to make write support for Agat840 disks? I've recently got Agat-9 back to life but unfortunately i don't have any disks to boot from :(
Thanks!
Yes, absolutely. The missing factor is having a real machine to test on, and you have one!
I've added preliminary support, but it's likely to be wrong. It's in the agat branch: #639 Could you take a look and let me know if it works, please?
Also, @dk1spb: I've just spotted your comments which I appear to have missed back in March?
There have been a lot of changes since then, including a massive rewrite to fluxengine-gui (it now contains a proper flux explorer --- see the 'explore disk' button!). Could you PTAL and tell me if the problem you were seeing is still occurring?
Sorry for the delay. I've discovered RAM problem at my Agat-9 and waiting for spare Soviet RAM chip to fix it so I can't check floppy right now. At the moment i've managed to successfully compile fluxengine from agat branch and wrote test floppy with Greaseweazle F1. I've used the same .dsk file i've used to create .hfe image which at least started to boot at my Agat before i've got RAM fail. .hfe was created using agath-to-hfe V2.zip and written out to floppy by the same Greaseweazle F1 I'll complete the test ASAP after fixing the hardware =-.-=
Regarding write disk: 1 GUI "Write Disk" button is blurred for "agat840" format 2 Writing at cylinders 75 and higher produce a lot of errors. May be some precompensation needed for higher tracks? The original Agat FDC uses precompensation 250ns starting from tracks 40+ and 500ns at tracks 60+
The Write Disk button works for me --- did you have the 'real disk' option set?
Regarding precompensation: oh dear. I've managed to avoid having to think about that much up until now. There is some unused and non-functional code for applying it but I always found it just made things worse; it's possible that I simply wasn't doing it right. I'll take another look, but I can't promise anything...
Right now while waiting for spare RAM I'm digging the FDC schematic and it's state machine ROMs. May be it will be not so difficult to recreate programmatically at least the original MFM encoder
I did some investigations including measurements and simulation using original schematics and original ROM contents and it seems I've got the original encoding algorythm. I'm not 100% sure as i didn't tested it on real hardware but still...
There is some catch regarding AIM data. It is FDC read data but not the FDC write commands sequence. To get FDC command sequence from AIM each SYN command in AIM should be converted to [ write A4 if not written in previous byte ] [ request syn + write FF ]
The encoder uses ~4MHz clock with 1 bit frame = 16 clock cycles. The actual measured clock is 245ns per pulse due to Agat clock generation system. The encoding algorythm is like this
byte_start:
buffer_reg = <FDC data byte>
if ( <FDC SYN command>) {
enc_reg = 0x12
wait( 8 )
}
for(bit=0; bit<8; bit++) {
delay = ROM[ 0x3F & enc_reg ]
if(delay < 15)
generate_mfm_pulse(delay) // total 16 clock cycles with MFM pulse at <delay> position
else
wait( 16 )
enc_reg << 1
enc_reg[0] = buffer_reg[7]
buffer_reg << 1
}
goto byte_start // next byte
Each encoder loop produce data pulse for current MFM bit interval and sync pulse for ongoing bit interval. Current processed data bit is enc_reg[3] so the encoded stream is delayed for 3 bit to new data byte load.
The write precompensation changes the encoding table using high 2 address bits at encoder ROM 00 - None, 0x80 - Enabled, 0xC0 - Strong
The decoded delay values ROM tables are (delay values are decimal, address is hex; X = no MFM pulse) No precomp:
0 1 2 3 4 5 6 7 8 9 A B C D E F
00 12 12 12 12 X X X X 4 4 4 4 4 4 4 4
10 12 12 12 12 X X X X 4 4 4 4 4 4 4 4
20 12 12 12 12 X X X X 4 4 4 4 4 4 4 4
30 12 12 12 12 X X X X 4 4 4 4 4 4 4 4
Enabled (tracks 40+):
0 1 2 3 4 5 6 7 8 9 A B C D E F
00 12 12 12 12 X X X X 4 4 4 4 4 4 4 4
10 12 12 12 12 X X X X 4 4 3 3 4 4 4 4
20 12 12 12 12 X X X X 4 4 4 4 5 5 5 5
30 12 12 12 12 X X X X 4 4 3 3 4 4 4 4
Strong (tracks 60+):
0 1 2 3 4 5 6 7 8 9 A B C D E F
00 12 12 11 11 X X X X 4 4 4 4 5 5 5 5
10 13 13 12 12 X X X X 3 3 2 2 4 4 4 4
20 12 12 11 11 X X X X 4 4 4 4 6 6 6 6
30 13 13 12 12 X X X X 3 3 2 2 4 4 4 4