micropython-ulab icon indicating copy to clipboard operation
micropython-ulab copied to clipboard

[BUG] loadtxt is sensitive to blank lines

Open Bzero opened this issue 11 months ago • 3 comments

Describe the bug loadtxt infers a wrong shape and omits values if the datafile contains blank lines.

ulab version: 6.7.2-2D

To Reproduce Create a data file with either of the following content (note the white space):

0 1

2 3

or

0 1
2 3


And try to load either of the files with loadtxt:

from ulab import numpy
print(numpy.loadtxt('data.dat'))

Both result in an array with too few columns and an omitted value:

array([[0.0],
       [1.0],
       [2.0]], dtype=float32)

Expected behavior

Blank lines should not make a difference.

Additional context

If the file contains more blank lines 0.0 is inserted, e.g. the file

0 1
2 3






results in the array

array([[0.0],
       [1.0],
       [2.0],
       [3.0],
       [0.0],
       [0.0],
       [0.0]], dtype=float32)

This issue may be related to #706.

Bzero avatar Jan 29 '25 23:01 Bzero

Yes, it's basically the same issue as https://github.com/v923z/micropython-ulab/issues/706.

v923z avatar Feb 01 '25 21:02 v923z

I actually regard this as a corner case, in fact, I would go as far as saying that having blank lines in a data file might be confusing, because it might mean multiple things.

It's not clear to me, either, whether you are concerned about linebreaks only, or anything that looks like an empty line, i.e., a line with multiple tabs and a linebreak etc.

I would vote for closing the issue.

v923z avatar May 29 '25 20:05 v923z

Generally I am concerned about any lines containing only whitespace. I agree that there are multiple ways a blank line in a data file could be interpreted, however I would still hold that omitting elements or adding zeros is not one of the ways that make much sense when it comes to handle blank lines. Of course one can just consider it an invalid datafile but for compatibility with numpy loadtxt I would still suggest to simply omit blank lines.

Bzero avatar Jun 29 '25 14:06 Bzero