Double expanded macros aren't parsed well for delayed invocation

[Tal500]

The following code isn't transformed well by simplecpp:

#define FOOBAR(x) x

#define MACRO_COMPOSE(A, B) A ## B

MACRO_COMPOSE(FOO, BAR(1))

gcc -E output:

1

simplecpp output:

FOOBAR ( 1 )

[danmar]

@Tal500 Please simplify this issue description. Please just show relevant gcc -E output.

[danmar]

I don't know.. this might be related to #397

[Tal500]

@Tal500 Please simplify this issue description. Please just show relevant gcc -E output.

I updated this issue now to contain simpler reproduction.

[danmar]

I updated this issue now to contain simpler reproduction.

Excellent :+1:

[Tal500]

I don't know.. this might be related to #397

I can confirm that the problem remains even though #397 was fixed.

[Tal500]

Looking on the code of simplecpp, the root cause of this issue as far as I can, is that simplecpp resolves function-like macros in a recursive way, instead of by passes.

From Wikipedia:

Order of expansion

Function-like macro expansion occurs in the following stages:

1. Stringification operations are replaced with the textual representation of their argument's replacement list (without performing expansion).
2. Parameters are replaced with their replacement list (without performing expansion).
3. Concatenation operations are replaced with the concatenated result of the two operands (without expanding the resulting token).
4. Tokens originating from parameters are expanded.
5. The resulting tokens are expanded as normal.

Meaning that according to the C/C++ rules, as far as I understand, you shall at each pass (viewing your source code as tokens):

• expand all the macros in your source code before expanding the macro-arguments (just substitute in all the relevant places)
• apply the quoting (#) or the concatenation (##) operator

and then repeat this on as many passes as needed until all the macros are resolved.

This "while loop" design is different than the simplecpp design of the macro expansion code. While I believe this logic simplifies simplecpp code drastically (assuming I'm right), I'm not sure where to start.

WDUT @danmar ?

[danmar]

@Tal500 I did not see such information when I started implementing simplecpp and therefore I implemented the macro expansion using trial and error. So it wasn't that I wanted to differ from the proper algorithm for any reason.

I would like that we refactor the expansion to use that proper algorithm instead. If we get good results then (I am pretty convinced that we do).

The algorithm should be described in some standard PDF somewhere, it would be preferable to point to the standard rather than wikipedia.

[Tal500]

@Tal500 I did not see such information when I started implementing simplecpp and therefore I implemented the macro expansion using trial and error. So it wasn't that I wanted to differ from the proper algorithm for any reason.

I would like that we refactor the expansion to use that proper algorithm instead. If we get good results then (I am pretty convinced that we do).

The algorithm should be described in some standard PDF somewhere, it would be preferable to point to the standard rather than wikipedia.

The best sources I recommend are from cppreference, containing both the C and the C++ (respectively) macro replacement rules: https://en.cppreference.com/w/c/preprocessor/replace https://en.cppreference.com/w/cpp/preprocessor/replace