Redesign compiler extensions to support adding the generated code to the file system

[azinneera]

Description:

In the current compiler extensions model, the generated code cannot be saved into the file system. The changes are kept in memory and added to the final executable or .bala file. We have use cases where the generated code should be persisted since the user needs this generated code to continue the rest of the development. Apart from this, there are several drawbacks that lead to thinking of a new design for compiler extensions. The identified drawbacks are as follows:

1. There are scenarios that need the generated code to be saved in the file system for further development E.g: Generation of client code based on an IDL file, generation of code for bal persist feature The current design is not extensible to support this. We have to keep providing new APIs to satisfy each use case
2. The plugin order matters due to the significant amount of changes a plugin can do to user code E.g: One plugin can generate additional services and another will add an annotation value to all services. If the latter runs first, the services generated in the former won’t have the annotation
3. Debugger user experience breaks E.g:
   • The generated sources are not persisted in the file system and so, step-in functionality will keep hitting the same line which is an unexpected behavior from the user PoV
   • Mismatch in source code and the byte-code
     • Modifiers may add new lines
     • Generators may add new files with services E.g: transformer, azure packages
4. Newly added code won’t be tested consciously by the user though they will be a part of the Ballerina program