components-js
components-js copied to clipboard
Add image attach feature to chat component
I have been wanting to implement an image upload feature for meet.livekit.io for some time now, and I finally got some time to hack on it! I also think this could be a good example that demonstrates the different use cases of Data Messages.
Currently, this feature would only support image uploading, but I believe down the road it could be expanded to multiple images and/or different file types.
Image Message Sending Architecture
- User uploads image
- Image data is divided into packets of 50k byte (LK size limit for data messages is ~65k I believe?)
- Each packet is sent as a LiveKit data message, with each message holding image packet properties like message id, packet index, total packet count and image data
Image Message Receiving Architecture
- Browser receives LiveKit data message which holds image properties
- Browser waits until all packets are arrived, ie. does not send individual packets to the user
- Once final packet is arrived, it creates an image blob concatenating all the image data
- The image blob is attached to a ChatMessage object and is send to the user
- If a message has an image attached to it, the component would display the image
Another way I considered implementing this feature was to create a new Observable object just for images, but I wasn't sure if that would be too much of an architectural change. I would also have to have two observables handling all incoming messages which maybe is not desired?
Test Findings I tested images with size up to 5MB and although they worked most of the time, I did find some rare instances where packet loss occurs. Images above 8 MB failed due to packet loss almost all the time. I am not sure if this is expected, even when the message mode is set to be reliable. I would love to hear if anyone have any insights on this! I set the maximum file size threshold at 3 MB (for now?) as in my testing, files lesser than 3 MB always went through fine and quick. I also tested concurrent file sending between two users in a room, and that seem to work as expected as well. I would like to probably test this feature with multiple users sending images concurrently to the room.
I would love to hear the team's and the community's thoughts, suggestions, concerns, bugs(if any) and test findings, and would love to make according changes! A sample demo of the feature can be found in this Slack message!