NetCoreServer icon indicating copy to clipboard operation
NetCoreServer copied to clipboard

Sending image bytes over TCP in LAN can send a packet with size over 65536

Open etheaven opened this issue 5 years ago • 5 comments

Proof: https://cdn.discordapp.com/attachments/328508701763698698/535807104565379073/unknown.png

sometimes it does not split the packet, I used the ChatSession example and modified it a little. I kept the files and code so I can provide it to you if needed

etheaven avatar Jan 18 '19 13:01 etheaven

TcpSessoin._receiveBuffer can become bigger than a single packet size. As the result Socket.ReceiveAsync() can receive bigger buffer at single operation. I think it is your case and you get OnReceived() handler called with size > 65536

chronoxor avatar Jan 18 '19 13:01 chronoxor

Also pay attention that your client should receive/consume data faster than server send/produce it. Otherwise some back-pressure logic have to be implemented on the top of server/client to limit buffer sizes. E.g. like in this sample - https://github.com/chronoxor/NetCoreServer/blob/master/performance/TcpMulticastServer/Program.cs

        public override bool Send(byte[] buffer, long offset, long size)
        {
            // Limit session send buffer to 1 megabyte
            const long limit = 1 * 1024 * 1024;
            long pending = BytesPending;
            if ((pending + size) > limit)
                return false;
            if (size > (limit - pending))
                size = limit - pending;

            return base.Send(buffer, offset, size);
        }

chronoxor avatar Jan 18 '19 13:01 chronoxor

TcpSessoin._receiveBuffer can become bigger than a single packet size. As the result Socket.ReceiveAsync() can receive bigger buffer at single operation. I think it is your case and you get OnReceived() handler called with size > 65536

well, that is correct? I mean.. If you did following in the OnReceived():

expectedSize += (int)size;
if (expectedSize != buffer.Length)
            {
                Console.WriteLine("Data desync");
            }else Console.WriteLine("Data sync OK");

then you would receive a data desync at some point - there must be a bug somewhere
and for the pending sender: doesn't socket code handle that?

etheaven avatar Jan 18 '19 15:01 etheaven

Send(Message1), Send(Message2) - the order will be preserved, but you may receive partial data as OnReceived(Message1.1), OnReceived(Message1.2), OnReceived(Message1.3), OnReceived(Message2.1), OnReceived(Message2.2). So the good way is to have upper level protocol, which sends message size as a first part of message. and then wait for size, wait for whole message and handle it.

chronoxor avatar Jan 18 '19 15:01 chronoxor

Greetings. Just want to quick open this again, and ask for maybe an addition to the Receive Async method. Im having a hard time right now to handle some bigger receiving operations as it cuts me of .

In another project of mine i used some similar and it quite worked out nice with this->

var bytesRead = 0;
                var bufferSize = 65536;
                var buffer = new byte[bufferSize];
                var bytesRemaining = args.ContentLength;
                Stream file = new FileStream(filePath, FileMode.Create);

                while (bytesRemaining > 0)
                {
                    bytesRead = args.DataStream.Read(buffer, 0, buffer.Length);

                    if (bytesRead > 0)
                    {
                        var consoleBuffer = new byte[bytesRead];
                        Buffer.BlockCopy(buffer, 0, consoleBuffer, 0, bytesRead);
                        file.Write(consoleBuffer, 0, bytesRead);
                    }

                    bytesRemaining -= bytesRead;
                }
            Maybe this would work out for your async receive method. 

Would help me out definitly

Crushedice avatar Apr 07 '20 21:04 Crushedice