Rendering system unification

[SiegeLord]

Current Allegro has two rendering systems, the one for drawing bitmaps inside core and the one for drawing everything else in the primitives addon. Their separation causes a number of complications in the implementation and makes it hard to implement certain features. This issue proposes moving the primitives' addon functions over to core, replacing core's drawing with the primitives addon's. This will yield the following improvements:

• Less code duplication, making it easier to effect improvements to the code, write new drivers etc.
• Allow us to have unified batching of drawing (al_hold_bitmap_drawing but for primitives as well)
• Simplify the implementation of the primitives addon's handling of certain things (mostly due to TLS)

[SiegeLord]

Proposed changes to the display etc (type names TBD, they might stay as ALLEGRO_ prefixed):

Remove:

• flush_vertex_cache
• prepare_vertex_cache

Add API from primitives addon:

• int draw_prim(const void* vtxs, const _AL_VERTEX_DECL* decl, ALLEGRO_BITMAP* texture, int start, int end, int type);
• int draw_indexed_prim(const void* vtxs, const _AL_VERTEX_DECL* decl, ALLEGRO_BITMAP* texture, const int* indices, int num_vtx, int type);
• int draw_vertex_buffer(_AL_VERTEX_DECL* vertex_buffer, ALLEGRO_BITMAP* texture, int start, int end, int type);
• int draw_indexed_buffer(_AL_VERTEX_DECL* vertex_buffer, ALLEGRO_BITMAP* texture, _AL_INDEX_BUFFER* index_buffer, int start, int end, int type);
• ALLEGRO_VERTEX_DECL* create_vertex_decl(const ALLEGRO_VERTEX_ELEMENT* elements, int stride);
• void _destroy_vertex_decl(_AL_VERTEX_DECL* decl);
• _AL_VERTEX_DECL* create_vertex_buffer(ALLEGRO_VERTEX_DECL* decl, const void* initi_data, int num_vertices, int flags);
• void, destroy_vertex_buffer(_AL_VERTEX_DECL* buffer);
• void* lock_vertex_buffer(_AL_VERTEX_DECL* buffer, int offset, int length, int flags);
• void unlock_vertex_buffer(_AL_VERTEX_DECL* buffer);
• int get_vertex_buffer_size(_AL_VERTEX_DECL* buffer);
• _AL_INDEX_BUFFER* create_index_buffer(int index_size, const void* initi_data, int num_indices, int flags);
• void destroy_index_buffer(_AL_INDEX_BUFFER* buffer);
• void* lock_index_buffer(_AL_INDEX_BUFFER* buffer, int offset, int length, int flags);
• void unlock_index_buffer(_AL_INDEX_BUFFER* buffer);
• int get_index_buffer_size(_AL_INDEX_BUFFER* buffer);

New batched drawing API:

• void set_vertex_batch_type(int type)
• void extend_vertex_batch(_AL_VERTEX *vertices, int count, int *indices, int count);
• void draw_vertex_batch()

The main complexity lies with Direct3D which has the concept of legacy cards (maybe we can remove it?) and also requires the use of shaders at all times. It would simplify things if we could switch to programmable pipeline to handle everything on Direct3D side. Perhaps we can.

[SiegeLord]

Roughly speaking, we'd support the following settings.

1. OpenGL 3.3 core (use vertex + index buffers + vao for the batch)
2. OpenGL <3.3 / GLES 1 (use CPU ALLEGRO_VERTEX arrays for the batch)
3. Software (use CPU ALLEGRO_VERTEX arrays for the batch)
4. Direct3D9 modern (use vertex + index buffers + for the batch + forced programmable pipeline)
5. DIrect3D9 legacy (use FVF vertices in CPU arrays, no shaders)

We'll remove the existing pre-compiled shaders from the primitives addon, and switch to Allegro's existing shaders by forcing the programmable pipeline.

[SiegeLord]

Not clear what to do about ALLEGRO_CFG_SHADER_HLSL, in some ways it was always wrong since Primitives addon used HLSL 100% of the time. Could perhaps treat it as going into the category (5).

[katastic]

Is there ever a scenario where you'd want software mode? Outside a niche project like "I'm writing my own CPU-only ray tracing engine in assembly"

[SiegeLord]

Maybe not, but it's an existing capability that isn't too hard to maintain.