Fix chunked message reassembly for out-of-order delivery
Problem
The existing chunking implementation had critical bugs that prevented markers and large CoT events from being transmitted successfully:
- Chunks were reassembled in arrival order, not logical order
- No message ID to separate concurrent messages
- No duplicate detection for retransmitted packets
- No timeout handling for incomplete messages
- Memory leaks from abandoned chunk data
This caused markers to fail silently when chunks arrived out of order over the mesh network, which is common due to multi-hop routing.
Solution
Implemented a robust chunking system with:
New Header Format
Changed from CHK_<size>_ to CHK_<msgId>_<chunkNum>_<totalChunks>_<totalSize>_
- Unique message ID separates concurrent transmissions
- Chunk numbering enables correct reassembly order
- Total size validates complete reassembly
MessageChunks Class
New helper class to track individual messages:
- Stores chunks in HashMap by chunk number
- Detects and filters duplicate chunks
- Reassembles in correct order regardless of arrival order
- Tracks message age for timeout detection
Enhanced ChunkManager
- Generates unique message IDs using Random
- Sends chunks with proper numbering (0-based)
- Reduced chunk size from 200 to 180 bytes (safer margin)
- Sends END marker with message ID
Improved MeshtasticReceiver
- Tracks up to 10 concurrent messages
- Automatic timeout cleanup (30 seconds)
- Drops oldest message when limit reached
- Comprehensive logging for debugging
Error Handling
- Invalid chunk numbers rejected
- Duplicate chunks filtered
- Timeout protection prevents memory leaks
- Resource limits prevent DoS
Testing
- All files compile without errors
- Backward incompatible (both sender and receiver must upgrade)
- Position sharing and text messages unaffected (different ports)
- Voice memos unaffected (different header)
- Did not test fully
Impact
- ✅ Markers now work over mesh networks
- ✅ File transfers more reliable
- ✅ Concurrent messages don't interfere
- ✅ Out-of-order delivery handled correctly
- ✅ Duplicate packets filtered
- ⚠️ Requires both users to upgrade for markers to work
Files Changed
- Constants.java: New header format and timeout constants
- ChunkManager.java: Message ID generation and new header
- MessageChunks.java: New class for tracking individual messages
- MeshtasticReceiver.java: Complete rewrite of chunk handling
Fixes issues with marker transmission over Meshtastic mesh networks.
Hi, I don't have an env to build this. is it possible for an owner to build this for me?
Hi, I don't have an env to build this. is it possible for an owner to build this for me?
How did you test it then?
Hi, I don't have an env to build this. is it possible for an owner to build this for me?
How did you test it then?
I wasn't able to fully test functionality