ATAK-Plugin icon indicating copy to clipboard operation
ATAK-Plugin copied to clipboard

Fix chunked message reassembly for out-of-order delivery

Open Dylanboyz opened this issue 1 month ago • 4 comments

Problem

The existing chunking implementation had critical bugs that prevented markers and large CoT events from being transmitted successfully:

  1. Chunks were reassembled in arrival order, not logical order
  2. No message ID to separate concurrent messages
  3. No duplicate detection for retransmitted packets
  4. No timeout handling for incomplete messages
  5. Memory leaks from abandoned chunk data

This caused markers to fail silently when chunks arrived out of order over the mesh network, which is common due to multi-hop routing.

Solution

Implemented a robust chunking system with:

New Header Format

Changed from CHK_<size>_ to CHK_<msgId>_<chunkNum>_<totalChunks>_<totalSize>_

  • Unique message ID separates concurrent transmissions
  • Chunk numbering enables correct reassembly order
  • Total size validates complete reassembly

MessageChunks Class

New helper class to track individual messages:

  • Stores chunks in HashMap by chunk number
  • Detects and filters duplicate chunks
  • Reassembles in correct order regardless of arrival order
  • Tracks message age for timeout detection

Enhanced ChunkManager

  • Generates unique message IDs using Random
  • Sends chunks with proper numbering (0-based)
  • Reduced chunk size from 200 to 180 bytes (safer margin)
  • Sends END marker with message ID

Improved MeshtasticReceiver

  • Tracks up to 10 concurrent messages
  • Automatic timeout cleanup (30 seconds)
  • Drops oldest message when limit reached
  • Comprehensive logging for debugging

Error Handling

  • Invalid chunk numbers rejected
  • Duplicate chunks filtered
  • Timeout protection prevents memory leaks
  • Resource limits prevent DoS

Testing

  • All files compile without errors
  • Backward incompatible (both sender and receiver must upgrade)
  • Position sharing and text messages unaffected (different ports)
  • Voice memos unaffected (different header)
  • Did not test fully

Impact

  • ✅ Markers now work over mesh networks
  • ✅ File transfers more reliable
  • ✅ Concurrent messages don't interfere
  • ✅ Out-of-order delivery handled correctly
  • ✅ Duplicate packets filtered
  • ⚠️ Requires both users to upgrade for markers to work

Files Changed

  • Constants.java: New header format and timeout constants
  • ChunkManager.java: Message ID generation and new header
  • MessageChunks.java: New class for tracking individual messages
  • MeshtasticReceiver.java: Complete rewrite of chunk handling

Fixes issues with marker transmission over Meshtastic mesh networks.

Dylanboyz avatar Nov 21 '25 18:11 Dylanboyz

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Nov 21 '25 18:11 CLAassistant

Hi, I don't have an env to build this. is it possible for an owner to build this for me?

Dylanboyz avatar Nov 21 '25 20:11 Dylanboyz

Hi, I don't have an env to build this. is it possible for an owner to build this for me?

How did you test it then?

niccellular avatar Nov 22 '25 19:11 niccellular

Hi, I don't have an env to build this. is it possible for an owner to build this for me?

How did you test it then?

I wasn't able to fully test functionality

Dylanboyz avatar Nov 22 '25 19:11 Dylanboyz