Common tables through a common internal service

[bruntib]

If CodeCompass will be extended with further language plugins, those will most probably be written in those languages. For example Java parser is written in Java. These should have access to the common tables like File or BuildAction. These common tables are described by ODB classes in C++. The question is how can other languages use these schemas?

One solution could be not having common tables. There could be a CppFile and JavaFile table. This idea might sound frightening, but after some personal discussions we found that it is not as horrible as it seems for the first sight.

However, there is another idea: we could implement an internal Thrift service which can be used during parsing and while web server is running, by the plugins. Inserting and reading records could be performed through this local service.

What do you think? Would it be a big overhead? Or are there other solutions on getting access to the common tables?

[mcserep]

Separate common tables (nice oxymoron) would introduce a couple issues hard to tackle I think. Just to mention a few:

• plugins working on common tables (like the search plugin) will have to deal with CppFile, JavaFile, etc. tables, their entity model written in different languages. Or will there be a SearchFile table, further escalating the redundancy?
• for projects containing source files written in multiple languages, simple file operations can get complicated, since e.g. for a file path it will have to be determined which XyFile table to look up in the first place?

I will be glad to read further opinions, but right now I think the few common tables (or at least File and FileContent) should be preserved to be common. A shared service managing these entities seems a more reasonable solution for me.

---

~~If we are dealing with such architectural refactoring, another issue would also worth consideration probably. Right now the CodeCompass_webserver serves the web frontend; and as a prototype, unmerged solution the code editors (through LSP). While CodeCompass is capable of incremental parsing and its LSP-compliant service can receive requests for reparsing files, the CodeCompass_webserver binary does not contains the parsers, so CodeCompass_parser have to be launched in an inconvenient way. It would be nice to have a service frontend which can receive requests to reparse files, and launch an incremental parsing appropriately.~~ Moved this part to a separate issue, see #350 .

[bruntib]

Yes, I know that this suggestion has quite a lot of drawbacks, but the current solution is having at least as many disadvantages. Me too, I don't think that it would be a final solution. I just thought of introducing this idea, because we had a brainstorming earlier, where we found that surprisingly this wouldn't be as bad as it seems. But let's forget it then :)

But let's discuss in the future about the other proposal, because common tables will be a problem for other languages plugins.

As for the other issue: maybe we should open another ticket about code edition in CodeCompass. I think this is a bigger topic. There are several other issues that may affect the "(incremental) parsing as a service" idea.

[mcserep]

We should warm up this topic @bruntib following the discussion last week.

[bruntib]

So in that discussion we mentioned that we already have this Thrift interface which handles the common tables: project.thrift. This should be extended with API functions which can write the common tables.

Currently C++ parser uses SourceManager class for handling source files: https://github.com/Ericsson/CodeCompass/blob/master/parser/include/parser/sourcemanager.h#L23 This class does some caching and other stuff, so we could consider using the solutions of this class in the implementation of the "common table writer" API functions in the future.

Nevertheless, I also have a concern here. In CodeChecker project the question about organizing the architecture comes up continuously. To be honest, I don't know what is the current status, but we always change our minds whether we want to hermetically separate the parser and the web service parts or not. Currently in CodeCompass we try to organize the code so the content of parser directory in the root is used by the parsers of the plugins only and the content of service directory is used by the plugin services only. Now this would be the first time that a parser plugin would use the common service directory.