Enable writing to a stream?

[Kequc]

I'm trying to write data to the stream as it comes in, but nothing seems to be getting sent from the client to the server. When I attach an 'on data' event listener on the client everything breaks. When I attach an 'on data' event listener on the server it is never triggered.

1. @bStream = ss.createStream()
2.
3. # if i add this i see an error
4. @bStream.on 'data', (data) ->
5.   console.log data.length
6.
7. ss(@socket).emit 'talk', @bStream, meta
8. @bStream.write myInt16ArrayBuffer

Error triggered on line 4 above.

Uncaught TypeError: Cannot read property '_read' of null
IOStream._read @ socket.io-stream.js:223
Readable.read @ socket.io-stream.js:3193
Readable.resume @ socket.io-stream.js:3562
Readable.on @ socket.io-stream.js:3528

I had everything working with Binaryjs, I wonder am I allowed to write to the stream like that with socket.io-stream?

[nkzawa]

Yes, you can write data like that, but stream#write only accepts Buffer or string, and objectMode is not supported for now.

[Kequc]

Thanks for your response, does an ArrayBuffer count as a Buffer? That's what I'm attempting to use.

[nkzawa]

ArrayBuffer is not Buffer, and I think you need browserify to use Buffer on browser. This is very confusing for users. We have to fix that :(

https://nodejs.org/api/buffer.html

[Kequc]

I wonder if a better solution would be to use socket.io without socket.io-stream and just send my array buffers to have them assembled for me on the server instead. Forgive me for not knowing this but the emits are sent in order if I do that correct? They cannot get all jumbled up?

Or should I change my stream#write calls with socket.io-stream to pass strings and then finagle them a bit on the server after that.

[nkzawa]

Socket.io keeps order of sending data. so you can just use socket.io, but splitting data to small chunks and combining them afterwards would be usually troublesome.

So basically, this is what socket.io-stream should support.

[nkzawa]

I exposed Buffer class for use on browser. https://github.com/nkzawa/socket.io-stream/commit/df60bcabd47359f2cb0594fafbfce6632883879d

It's just Buffer on node, and you can use it like:

sream.write(new ss.Buffer([1,2,3,4,5]));

[Kequc]

I'm making an effort to convert my typed array into Buffer before writing it to the stream. For some reason I don't think Buffer is turning up properly client side.

1. l = myInt16.byteLength
2. buffer = new ss.Buffer l
3.
4. while l--
5.   buffer[l] = myInt16[l]
6.
7. buffer

returns:

Uncaught TypeError: undefined is not a function

on line 2 above.

I think I'm compiling properly from console.

cd node_modules/socket.io-stream
browserify index.js -s ss > socket.io-stream.js
subl socket.io-stream.js

Then I copy and paste the contents of that file into my client side javascript. I see your change in there.

exports.Buffer = Buffer

I'm not sure what's wrong.

[nkzawa]

Sorry, it's not released yet. if you'd like to try, change dependencies on package.json like "socket.io-stream": "nkzawa/socket.io-stream#master",

[Kequc]

You're right. I was mistaken, I got that working... but... :(

5::/microphone:{"name":"$stream-read","args":["ba4d0198-d786-494a-b31b-43bff29d4006",16384]}
5:1+:/microphone:{"name":"$stream-write","args":["ba4d0198-d786-494a-b31b-43bff29d4006",{},"buffer"]}
5::/microphone:{"name":"$stream-error","args":["9b57c954-ac95-488d-9c20-7919a8ec3ef6","Invalid non-string/buffer chunk"]}

It's not your fault, this whole week has been a bust so far.

[Kequc]

# @bStream = ss.createStream()

_onAudio: (e) =>
  # Audio process

  data = e.inputBuffer.getChannelData 0

  @bStream.write @_convertInt16toBuffer @_convertFloat32ToInt16 data

_convertFloat32ToInt16: (buffer) =>
  # Convert buffer to 16 bit

  l = buffer.length
  buf = new Int16Array l

  while l--
    s = Math.max(-1, Math.min(1, buffer[l]))
    buf[l] = s * (if s < 0 then 0x8000 else 0x7FFF)

  buf.buffer

_convertInt16toBuffer: (buffer) =>
  # Convert buffer to stream compatible buffer

  buf = new ss.Buffer buffer.byteLength
  l = buf.length

  while l--
    buf[l] = buffer[l]

  buf

As a contrast, this all works perfectly sans the _convertInt16toBuffer method, with binaryjs, I'd like to move away from binaryjs. It seems socket.io-stream just doesn't quite like my ss.Buffer generated buffer somehow.

I've also tried returning buf rather than buf.buffer from _convertFloat32ToInt16 with the same result.

[nkzawa]

Hmm, I don't know why. As far as I tested, it seems to work well. Can I ask what version of browserify do you use?

[Kequc]

[email protected] /usr/local/lib/node_modules/browserify

[nkzawa]

It looks ok. Sorry, I have no idea ... The code I tested is the following.

var stream = ss.createStream();
ss(socket).emit('foo', stream);
stream.write(new ss.Buffer([0, 1]));
stream.write(new ss.Buffer([2, 3]));
stream.end();

Works fine with Browserify 9.0.8, on Chrome. What's the difference?

[Kequc]

I dunno man, I'm emitting just the same way. I double checked my convert from arraybuffer to buffer code from somewhere else on the net. I suppose I'm just generating an invalid arraybuffer... or who knows what else.

I'm really sorry I'm not more help. I think for now my implementation doesn't work and I'll need to hack with someone on it. Peer programming in my future.

[wyvernbai]

@Kequc Below code can get the stream from client. But I can not store the stream as audio file. Do you have any idea?

Server code:

ss(socket).on('stream_start', function(stream, meta) {
        filename = "recordings/"+ new Date().getTime()  + ".pcm";
        fileWriter = fs.createWriteStream(filename);
        stream.pipe(fileWriter);
    });
    ss(socket).on('close', function() {
        if (fileWriter != null) {
            fileWriter.end();
                 }
    });

Client code:

function onAudio(e) {
        if(!bStream || !bStream.writable) 
        {
            return;
        };

        var left = e.inputBuffer.getChannelData(0);
        var canvas = document.getElementById("canvas");
        drawBuffer( canvas.width, canvas.height, canvas.getContext('2d'), left );
        console.log("onAudio");
        reSample(e.inputBuffer,resampleRate,function(reSampledBuffer){
            var channel = reSampledBuffer.getChannelData(0);
            bStream.write(convertFloat32ToInt16(channel));
        });

    }

function convertFloat32ToInt16(buffer) {
        var l = buffer.length;
        var buf = new Int16Array(l);
        var return_buf = new ss.Buffer(l);
        while (l--) {
            buf[l] = Math.min(1, buffer[l])*0x7FFF;
            return_buf[l] = buf[l];
            if (l % 100 == 0){
                console.log(return_buf[l]);
            }
        }
        return return_buf;
    }

[ainsleys]

@wyvernbai were you ever able to get it working? Got about as far as you, but not able to save as a .wav

[Kequc]

My implementation that works has me sending base64 encoded data (strings) in chunks to the server, then decoding base64 and writing it to a stream. I'm using a version of this library: https://github.com/chris-rudmin/Recorderjs which encodes my audio into the opus audio codec. The library seems to be fluctuating currently, so it is changing, but it works well to ensure I'm sending a smaller amount of data than wav would require.

Then on the server I simply put it together using stream.write. Sorry I haven't updated this topic sooner. I thought the topic would have been out of date. Instead of using socket.io-stream I'm just sending raw strings.

Edit: Specifically I'm using my fork. Which is quite out of date by now, but in case it is useful: https://github.com/Kequc/Recorderjs

[ainsleys]

Ahh, that's a good approach. Thanks for the detailed response. I will try your method!

On Mon, Aug 8, 2016 at 2:45 PM, Kequc [email protected] wrote:

My implementation that works has me sending base64 encoded data (strings) in chunks to the server, then decoding base64 and writing it to a stream. I'm using a version of this library: https://github.com/chris- rudmin/Recorderjs which encodes my audio into the opus audio codec. The library seems to be fluctuating currently, so it is changing, but it works well to ensure I'm sending a smaller amount of data than wav would require.

Then on the server I simply put it together using stream.write. Sorry I haven't updated this topic sooner. I thought the topic would have been out of date. Instead of using socket.io-stream I'm just sending raw strings.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nkzawa/socket.io-stream/issues/44#issuecomment-238387807, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMLOWdzuegxgIQkv8PKz05NxtVGJJPWks5qd6OUgaJpZM4EES2u .