pointer aliasing/alignment issue in rfbSendSetColourMapEntries()

[ReBeating]

Description

The function rfbSendSetColourMapEntries contains a pointer aliasing and memory alignment issue. This issue may lead to crashes or performance degradation on certain platforms, such as those requiring strict memory alignment (e.g., ARM architectures).

Location

File Path: src/libvncserver/rfbserver.c
Function Name: rfbSendSetColourMapEntries
Code Snippet:

char buf[sz_rfbSetColourMapEntriesMsg + 256 * 3 * 2];
char *wbuf = buf;
rfbSetColourMapEntriesMsg *scme;
uint16_t *rgb;

if (nColours > 256) {
    wbuf = (char *) malloc(sz_rfbSetColourMapEntriesMsg + nColours * 3 * 2);
}

scme = (rfbSetColourMapEntriesMsg *)wbuf;
rgb = (uint16_t *)(&wbuf[sz_rfbSetColourMapEntriesMsg]);

Impact

Pointer Aliasing Issue: The code uses a char array and casts it to rfbSetColourMapEntriesMsg and uint16_t pointers. This may result in unaligned memory access.

Fix

Referencing the patch file commit.patch, the issue can be resolved by:

1. Using a union to replace the char array and avoid unsafe typecasting.
2. Ensuring proper memory alignment for all accesses.

Ref

This is similar to CVE-2020-14400. Link: https://github.com/LibVNC/libvncserver/commit/53073c8d7e232151ea2ecd8a1243124121e10e2d

[bk138]

Thanks for reporting, please also file a PR with the fix you're mentioning (no commit.patch found in this post) :-)