Different format for symbol?

[kennethlimcp]

Ran into an issue where some symbols cannot be processed, throwing this error:

Traceback (most recent call last):
  File "/Users/kennethlimcp/Desktop/read-ms/ms2txt/metastock/files.py", line 23, in dump
    self.load_columns()
  File "/Users/kennethlimcp/Desktop/read-ms/ms2txt/metastock/files.py", line 46, in load_columns
    assert(len(lines) == self.stock.fields)
AssertionError
Processing FTBM (fileNo 2)
('Error while converting symbol', u'FTBM')

Dumping the info gives: (8, 7, ['"DATE",0,0', '"OPEN",5,5', '"HIGH",5,5', '"LOW",5,5', '"CLOSE",5,5', '"VOL",0,0', '"OI",0,0', '\x1a'])

It seems like there is an extra column with data '\x1a' that is causing the length assertion.

Any idea what that might be?

[themech]

DOP is supposed to be an ASCII file and 0x1a is very suspicious. I will try to take a look at the format description I was using (http://xelay.narod.ru/msfl.html) and try to figure it out. Maybe it is a corrupted file or a different version of the format I was not aware of. Is the metastock data file you're using available somewhere so I could check it myself and do some debugging?

[themech]

@kennethlimcp after giving it some more thought - the DOP file does not look ok. This is a sample DOP file for 7 column Metastock data from my broker

"DATE",0,0
"OPEN",3,0
"HIGH",3,0
"LOW",3,0
"CLOSE",3,0
"VOL",0,0
"OI",0,0

As you see it has 7 lines, each describing one column. The '\x1a' looks like an extra line with some strange character, maybe it should just be ignored.