Menu overlaps drawable area after al_set_display_menu() on Windows with OpenGL.

[NewCreature]

While trying to solve #1431, I found an issue with OpenGL on Windows where, after adding a menu to a display with al_set_display_menu(), the menu overlapped the drawable area of the display. The issue goes away when the display is resized with al_resize_display() or by dragging on the window borders.

Additionally, minimizing the window and bringing it back can cause the same kind of bug with slightly different positioning.

The bug manifests differently on different machines. On the first machine on which I noticed this, the overlap issue happens immediately and I can trigger the minimize bug as well. Sometimes the drawable area will stick lower than it's supposed to be when removing the menu. On another machine, I can only trigger the minimize bug, but not the initial full overlap. In both cases, the same set of actions consistently produces the same results.

Here's a test program I made for this issue:

#include <allegro5/allegro.h>
#include <allegro5/allegro_font.h>
#include <allegro5/allegro_native_dialog.h>
#include <stdio.h>

int main(int argc, char * argv[])
{
	ALLEGRO_DISPLAY * display = NULL;
	ALLEGRO_MENU * top_menu = NULL;
	ALLEGRO_MENU * menu = NULL;
	ALLEGRO_FONT * font = NULL;
	ALLEGRO_EVENT_QUEUE * queue = NULL;
	ALLEGRO_EVENT event;
	int height[3] = {0};
	int i;

	if(!al_init())
	{
		goto fail;
	}
	if(!al_install_keyboard())
	{
		goto fail;
	}
	if(!al_init_native_dialog_addon())
	{
		goto fail;
	}
	if(!al_init_font_addon())
	{
		goto fail;
	}
	al_set_new_display_flags(ALLEGRO_OPENGL);
	display = al_create_display(800, 600);
	if(!display)
	{
		goto fail;
	}
	height[0] = al_get_display_height(display);
	font = al_create_builtin_font();
	if(!font)
	{
		goto fail;
	}
	queue = al_create_event_queue();
	if(!queue)
	{
		goto fail;
	}
	al_register_event_source(queue, al_get_display_event_source(display));
	al_register_event_source(queue, al_get_keyboard_event_source());
	menu = al_create_menu();
	if(!menu)
	{
		goto fail;
	}
	al_append_menu_item(menu, "Item 1", 0, 0, NULL, NULL);
	al_append_menu_item(menu, "Item 2", 1, 0, NULL, NULL);
	top_menu = al_create_menu();
	if(!top_menu)
	{
		goto fail;
	}
	al_append_menu_item(top_menu, "Main", 2, 0, NULL, menu);
	al_set_display_menu(display, top_menu);
	top_menu = NULL;

	while(1)
	{
		al_get_next_event(queue, &event);
		if(event.type == ALLEGRO_EVENT_DISPLAY_CLOSE)
		{
			break;
		}
		else if(event.type == ALLEGRO_EVENT_KEY_DOWN)
		{
			height[1] = al_get_display_height(display);
			if(!top_menu)
			{
				top_menu = al_remove_display_menu(display);
			}
			else
			{
				al_set_display_menu(display, top_menu);
				top_menu = NULL;
			}
			height[2] = al_get_display_height(display);
		}
		al_rest(1.0 / 60.0);
		al_clear_to_color(al_map_rgb(0, 0, 0));
		for(i = 0; i < 3; i++)
		{
			al_draw_textf(font, al_map_rgb(255, 255, 255), 0, i * al_get_font_line_height(font), 0, "%d", height[i]);
		}
		al_flip_display();
	}
	al_destroy_font(font);
	al_destroy_event_queue(queue);
	al_destroy_display(display);

	return 0;

	fail:
	{
		if(top_menu)
		{
			al_destroy_menu(top_menu);
		}
		if(menu)
		{
			al_destroy_menu(menu);
		}
		if(queue)
		{
			al_destroy_event_queue(queue);
		}
		if(font)
		{
			al_destroy_font(font);
		}
		if(display)
		{
			al_destroy_display(display);
		}
		return -1;
	}
}

[SiegeLord]

Allegro sends a display resize event that, if handled, makes things work for me. It just so happens that for Direct3D that doesn't appear to do anything, but is necessary for OpenGL (perhaps some transformation difference). Not sure if this is expected (although the docs do talk about this resize event). It's a bit weird that we're sending it even if the display is not resizable.

[SiegeLord]

I think the doc is wrong, we don't actually send a resize event when creating the menu like the documentation claims. We have a mechanism to suppress that resize event, but evidently we need to also suppress it while maximizing (and maybe other situations?). Needs more investigation: need to check that recent change to change how we compute the display size possibly as well.

[SiegeLord]

The minimize part of this bug turned out to be unrelated to menus or OpenGL. There was some code that was improperly setting window flags after restoring the window.

[SiegeLord]

I'm gonna keep this open, since OP said there was an issue that happened even without minimizing.