Stdf file writing issue

[mgoldste1]

I know you said at one point writing STDF files with this library was kinda iffy. I was just using it to create some stdf files for some unit tests and I found an stdf file writing bug. I figured I'd document it here.

Test case for it:

        [TestMethod]
        public void ByteArrayRT_EDITED()
        {
            StdfFile f = new(@"file1.stdf");
            f.IndexingStrategy = new V4StructureIndexingStrategy();

            var mprs_file1 = f.GetRecords().OfExactType<Mpr>().ToList();
            byte[] pinList = { 0,0,0,0,0,0,0,0,0,
                               1,1,1,1,1,1,1,1,1,
                               2,2,2,2,2,2,2,2,2,
                               3,3,3,3,3,3,3,3,3,
                               4,4,4,4,4,4,4,4,4,
                               5,5,5,5,5,5,5,5,5,
                               6,6,6,6,6,6,6,6,6,
                               7,7,7,7,7,7,7,7,7,
                               8,8,8,8,8,8,8,8,8,
                               9,9,9,9,9,9,9,9,9
                             };
            float[] results1 = new float[90];
            ushort[] indexes = new ushort[90];

            for (int i = 0; i < 90; i++)
            {
                results1[i] = 100f + i;
                indexes[i] = mprs_file1[0].PinIndexes![i];
            }

            mprs_file1[0].PinIndexes = indexes;
            mprs_file1[0].Results = results1;
            mprs_file1[0].PinStates = pinList;

            StdfFileWriter sfw = new(@"file2.stdf");
            sfw.WriteRecords(f.GetRecordsEnumerable());
            sfw.Dispose();

            StdfFile f2 = new(@"file2.stdf");
            f2.IndexingStrategy = new V4StructureIndexingStrategy();
            var mprs_file2 = f2.GetRecords().OfExactType<Mpr>().ToList();

            //the Assert.AreEqual was failing on the arrays so we are looping through it.
            for(int i=0; i<90; i++)
                Assert.AreEqual(mprs_file1[0].Results![i], mprs_file2[0].Results![i]); //passes
            for (int i = 0; i < 90; i++)
                Assert.AreEqual(mprs_file1[0].PinIndexes![i], mprs_file2[0].PinIndexes![i]); //passes
            for (int i = 0; i < 90; i++)
                Assert.AreEqual(mprs_file1[0].PinStates![i], mprs_file2[0].PinStates![i]); //fails
        }

The last assert fails. When you write a byte array, at least in an MPR record (didnt check elsewhere), the order of the bytes change. The pin lists I entered starts with 9 zeros. After it's written, the order changes to the following:

 9, 9, 9, 9, 9, 9, 9, 9, 8,
 9, 8, 8, 8, 8, 8, 8, 8, 8,
 7, 7, 7, 7, 7, 7, 7, 7, 6,
 7, 6, 6, 6, 6, 6, 6, 6, 6,
 5, 5, 5, 5, 5, 5, 5, 5, 4,
 5, 4, 4, 4, 4, 4, 4, 4, 4,
 3, 3, 3, 3, 3, 3, 3, 3, 2,
 3, 2, 2, 2, 2, 2, 2, 2, 2,
 1, 1, 1, 1, 1, 1, 1, 1, 0,
 1, 0, 0, 0, 0, 0, 0, 0, 0

I verified it with a different stdf file reader.

This doesn't impact me because I rarely write stdf files, but it could potentially cause misprocessing for others. It would be nice to be able to use this package to script changes to stdf files though.

BTW, my stdf project is finally coming to an end so once things calm down a little, I'll start working on getting the approvals to actually contribute here. I've never needed to go through this process before though so I don't know what requirements they have for it to be approved.

[marklio]

Thanks for finding and documenting this fix! I can totally imagine this bug existing. Due to the way that missing data is represented in STDF (records can be truncated at the point where no remaining fields contain data), record fields are traversed from the last to the first and are written to memory "backwards" and then reversed before being written to disk. I'll bet the logic for arrays is just broken (and probably always has been), and needs to traverse arrays backwards while writing in order to get the right order. This is probably an easy fix as well.

[mgoldste1]

The weird thing is that it isn't just backwards. If it was just backwards, the 9th item would be a 9 and the 10th would be an 8, but its the opposite. This happens every other time the number changes.

edit: made the wording more clear

[marklio]

oh, weird. I guess it needs more investigation than a guess :)

[dingetje]

pinstates in MPR records are nibbles, and there's this weird padding for odd/even thing too. Maybe that's what wrong here?

RTN_ICNT, RTN_INDX, RTN_STAT The number of element in the RTN_INDX and RTN_STAT arrays is determined by the value of RTN_ICNT. The RTN_STAT field is stored 4 bits per value. The first value is stored in the low order 4 bits of the byte. If the number of indexes is odd, the high order 4 bits of the last byte in RTN_STAT will be padded with zero. The indexes referred to in the RTN_INDX are the PMR indexes defined in the Pin Map Record (PMR). The return state codes are the same as those defined for the RTN_STAT field in the FTR. RTN_ICNT may be omitted if it is zero and it is the last field in the record.