Automatic API conversion

[slouken]

It would be really handy to have a script that will automatically rename headers, constants, API functions (and reorder arguments as needed) that we can include with an SDL3 migration guide.

It might even be beneficial to have some kind of metadata that describes the changes that could be parsed and used in sdl2-compat as well as this script?

I'm also thinking that we should standardize on Python 3 as the scripting for public facing scripts like these, since it's got a clean and easy to read syntax, is widely available on platforms, and is widely taught in programming curriculums.

Thoughts?

@libsdl-org/a-team

[1bsyl]

A python3 script would be really convenient for (window) users.

But, here's a bad case, if you want to change int foo(int a, char *b); to bar(char *b, int a);

in something like :

foo(
   foo(2,"foo"), ""
);

You need a strong python script with recursive regex... probably a bad example. but at least it needs to handle: multi-lines and nested calls in the arguments.

If you choose some tool out of the shelve like https://github.com/coccinelle/coccinelle (used in linux kernel) you just have to write 5 lines and that's done:

@@
expression a, b;
@@
- foo(a, b)
+ bar(b, a)

[madebr]

I've written a clang-tidy plugin at https://github.com/madebr/sdl2to3. It currently has limited usage and probably contains dozens of bugs. It warns about removed functions, converts SDL_CreateRGBSurface to SDL_CreateSurface and it converts #include's. But it is a start.

It needs llvm 15.0 because earlier versions do not contain headers to create clang-tidy plugins.

Running it on the following source,

#include "SDL.h"
#include "SDL_stdinc.h"

#include "SDL2/SDL_syswm.h"

int an_arg(int a) {
    return 3;
}

void func() {
    SDL_CalculateGammaRamp(0, (void*)0);
    SDL_CreateRGBSurface(an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5), an_arg(6), an_arg(7));
    SDL_CreateRGBSurfaceWithFormat(an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4));
    SDL_CreateRGBSurfaceWithFormatFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5));
    SDL_CreateRGBSurfaceFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5), an_arg(6), an_arg(7), an_arg(8));
}

gives the following output:

14 warnings generated.
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:1:11: warning: SDL2-style file inclusion is deprecated. [sdl2to3-includefolder]
#include "SDL.h"
 ~~~~~~~~~^~~~~~
 include <SDL3/SDL.h>
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:2:11: warning: SDL2-style file inclusion is deprecated. [sdl2to3-includefolder]
#include "SDL_stdinc.h"
 ~~~~~~~~~^~~~~~~~~~~~~
 include <SDL3/SDL_stdinc.h>
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:4:11: warning: SDL2-style file inclusion is deprecated. [sdl2to3-includefolder]
#include "SDL2/SDL_syswm.h"
 ~~~~~~~~~^~~~~~~~~~~~~~~~~
 include <SDL3/SDL_syswm.h>
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:11:5: warning: This function is removed from SDL3. [sdl2to3-removedfunctions]
    SDL_CalculateGammaRamp(0, (void*)0);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:12:5: warning: SDL_CreateRGBSurface is removed from SDL3, and can be replaced by SDL_CreateSurface. [sdl2to3-SDL_CreateRGBSurface]
    SDL_CreateRGBSurface(an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5), an_arg(6), an_arg(7));
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    SDL_CreateSurface(an_arg(1), an_arg(2), SDL_MasksToPixelFormatEnum(an_arg(3), an_arg(4), an_arg(5), an_arg(6), an_arg(7)))
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:13:5: warning: SDL_CreateRGBSurfaceWithFormat is removed from SDL3, and can be replaced with SDL_CreateSurface. [sdl2to3-SDL_CreateRGBSurfaceWithFormat]
    SDL_CreateRGBSurfaceWithFormat(an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4));
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    SDL_CreateSurface(an_arg(1), an_arg(2), an_arg(4))
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:14:5: warning: SDL_CreateRGBSurfaceWithFormatFrom is removed from SDL3, and can be replaced with SDL_CreateSurfaceFrom. [sdl2to3-SDL_CreateRGBSurfaceWithFormatFrom]
    SDL_CreateRGBSurfaceWithFormatFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5));
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    SDL_CreateSurfaceFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(4), an_arg(5))
/tmp/tmp.Mh3SyMy9Yn/src/tests/all.c:15:5: warning: SDL_CreateRGBSurfaceFrom is removed from SDL3, and can be replaced with SDL_CreateSurfaceFrom. [sdl2to3-SDL_CreateRGBSurfaceFrom]
    SDL_CreateRGBSurfaceFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(3), an_arg(4), an_arg(5), an_arg(6), an_arg(7), an_arg(8));
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    SDL_CreateSurfaceFrom((void*)an_arg(0), an_arg(1), an_arg(2), an_arg(4), SDL_MasksToPixelFormatEnum(an_arg(3), an_arg(5), an_arg(6), an_arg(7), an_arg(8)))
Suppressed 6 warnings (6 with check filters).

[1bsyl]

Updating also my stuff. I've written (and generated) the rules to convert ARGB masks into pixel_format. This gives me this script to update SDL_image SDL_migration_tmp.cocci.txt

example of rules :

// clear 'flags' (e1) and set 'depth´ to 0
// match hard-coded mask to format ARGB2101010
// function renaming
 - SDL_CreateRGBSurface(e1, e2, e3, 32, 0x3FF00000, 0x000FFC00, 0x000003FF, 0xC0000000)
 + SDL_CreateRGBSurfaceWithFormat(0, e2, e3, 0, SDL_PIXELFORMAT_ARGB2101010)

or like:

// find that we use the masks from format "e4->format"
 - SDL_CreateRGBSurface(e1, e2, e3, e4->BitsPerPixel, e4->Rmask, e4->Gmask, e4->Bmask, e4->Amask)
 + SDL_CreateRGBSurfaceWithFormat(0, e2, e3, 0, e4->format)

or

// last rule, if none applied, fallback to use SDL_MasksToPixelFormatEnum()
-SDL_CreateRGBSurfaceFrom(e1, e2, e3, e4, e5, e6, e7, e8, e9)
 +SDL_CreateRGBSurfaceWithFormatFrom(0, e2, e3, 0, e5, SDL_MasksToPixelFormatEnum(e4, e6, e7, e8, e9))
 

To try to use:

# apt-get install coccinelle
spatch --sp-file path/to/SDL_migration.cocci . > your_diff.txt
patch -p1 < your_diff.txt

A diff was generated for SDL_image. https://github.com/libsdl-org/SDL_image/commit/bb9d216a3b25a0819a450f20349fceeea7bf4c9d

This needed some manual rewrite because of the ifdef Big/Little endian not done for both cases. And it didn't patch all functions in the best way: it used the fallback "SDL_MasksToPixelFormatEnum" when this could be done with a constant format.

But in the end, it's kind of efficient...

[slouken]

@madebr, @1bsyl, do either of you want to keep up with the SDL3 API changes with your proposed automation? It seems like the coccinelle tool is a better one for actually converting SDL2 code, rather than just providing warnings, but ...?

[1bsyl]

I can propose some coccinelle file to see if that goes well. I believe it can handle most of basic renaming / permutation of argument. may be hard for complex situations

[1bsyl]

so first shot: https://github.com/libsdl-org/SDL/pull/7042

// - all function renaming (without parameter change) as in SDL_oldnames.h // - remove SDL_WINDOW_SHOWN // - migrate SDL_CreateRGB* and SDL_ConvertSurface* functions

[slouken]

Would it handle renaming structure members if we end up doing that?

[1bsyl]

yes, it does that quite easily:

   @@
   SDL_Surface *surf;
   @@
   - surf->format
   + surf->foo

also need to have

   @@
   SDL_Surface surf;
   @@
   - surf.format
   + surf.foo

[slouken]

Nice :)

[madebr]

A clang-tidy plugin is a nice idea, but not that easy to implement as @1bsyl's cocci script. Let's go all-in on the coccinelle script.

[slouken]

We have finalized on coccinelle as our main API renaming tool. We also have build-scripts/rename_symbols.py for arbitrary renaming in source code.