The input record separator on method IO.each is changed when nmatrix library in required

[seoanezonjic]

On ruby version ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux] when nmatrix is required, the following code:

` File.open(file).each do |line|

        line.chomp!
            puts line + '-'
            fields = line.split("\t")
            puts fields.inspect
            records << fields.map(&:to_f)
    end

Generates this output:

c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 b1 1 1 1 1 1 1 1 1 1 0 b2 1 1 1 1 1 1 1 1 1 0 b3 0 0 0 0 0 0 0 0 0 0 b4 0 0 0 0 0 0 0 0 0 0 b5 0 0 0 0 0 0 0 0 0 0 b6 0 0 0 0 0 0 0 0 0 0 b7 0 0 0 0 0 0 0 0 0 0 b8 0 0 0 0 0 0 0 0 0 0 b9 0 0 0 0 0 0 0 0 0 0 b10 0 0 0 0 0 0 0 0 0 0- ["c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10\nb1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0\nb2", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0\nb3", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb4", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb5", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb6", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb7", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb8", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb9", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb10", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]`

If I change the each line to: File.open(file).each(sep=$/) do |line| The each method has its normal behaviour and I get this ouput: c1 c2 c3 c4 c5 c6 c7 c8 c9 c10- ["c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10"] b1 1 1 1 1 1 1 1 1 1 0- ["b1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0"] b2 1 1 1 1 1 1 1 1 1 0- ["b2", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0"] b3 0 0 0 0 0 0 0 0 0 0- ["b3", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b4 0 0 0 0 0 0 0 0 0 0- ["b4", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b5 0 0 0 0 0 0 0 0 0 0- ["b5", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b6 0 0 0 0 0 0 0 0 0 0- ["b6", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b7 0 0 0 0 0 0 0 0 0 0- ["b7", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b8 0 0 0 0 0 0 0 0 0 0- ["b8", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b9 0 0 0 0 0 0 0 0 0 0- ["b9", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] b10 0 0 0 0 0 0 0 0 0 0- ["b10", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"] [[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]

Why nmatrix changes the IO.each method? Thanks in advance

[translunar]

Honestly, I'm not sure. I don't see where it is changing IO.each, but it's been a long time since I messed with any of the IO stuff. I'd suggest starting by looking in lib/nmatrix/io, probably.

[moloh]

@mohawkjohn I was hit by this and investigated a little. It looks like this is result of using Packable gem (https://github.com/marcandre/packable/blob/master/lib/packable/extensions/io.rb). That gem is doing some extensive monkey patching and overwrites number of core methods. Is it possible to strip that gem?

[translunar]

Unfortunately, packable is pretty necessary for the MATLAB-format matrix readers and writers. If someone wanted to rewrite those, it'd be fine to shed packable.

@moloh Please open an issue here: https://github.com/sciruby/packable/issues

[v0dro]

Close this? Doesn't seem related to nmatrix.

[translunar]

@v0dro I think the idea is that we probably need to come up with a solution in NMatrix that doesn't require packable. Or perhaps that we should make Matlab IO into a plugin.

[v0dro]

I like the plugin idea. Its not a core functionality anyway.

[ralampay]

Is there a fix for this already? Can't run my neural network implementation using nmatrix's latest version due to the packable gem. :(

[translunar]

No. We're all volunteers. I'd encourage you to submit whatever workaround you come up with.

[seoanezonjic]

Hi all I'm using nmatrix again and this problem seems that is gone. When I have revised the packable gem repo, there are three commits in Sep 20, 2018 that seems to fix this issue. I did some tests with the initial data and I cannot reproduce the issue. Maybe it would be nice to close this issue. Thank by you help