Redesign the Implementation

[git-afsantos]

A note to self that the current implementation needs a serious rework from the ground up, whenever I find enough available time.

metamodel.py: Change all data structures to hold IDs instead of references to other objects. As convenient as references are, testing the current system is harder than it should be. This will have an impact in all the code base, as well as in the query language.

haros.py: Change the commands/runners to run on an event and plug-in system. Make everything (extractors, query engine, analyser, exporter, etc.) a plug-in, so that it becomes truly customisable and extensible.

config_builder.py: This is a mess from its complexity alone, but the code needs a rewrite and clean up.

extractor.py: See above.

viz: All of it is asking for improvement and more user interaction.

[git-afsantos]

Note on plugins:

• eliminate the current haros_plugins repository
• make each plugin an independent package, with its own requirements and version control
• install and discover plugins with setuptools entry points
• keep the .haros/plugins directory (initially empty) for users to write their own non-distributed plugins; possibly load these using PluginBase
• could use different entry points for different types of plugins (analysis, extraction, export, ...)

---

@gavanderhoorn is the approach above for plugins ok with you?

may I suggest to try to avoid growing Haros into a pkg manager for plugins? Where would the plugins be installed to? Would it still use pip? In that case a virtualenv makes a lot of sense.

I'm currently working on packaging haros for distribution (and use) in a binary/compiled ROS environment, and I'm not sure having Haros install plugins on its own is making that easier.

In this case, distributed plugins would be installed independently via pip, e.g. pip install haros-cppcheck

This would fit nicely into a virtualenv, as you say, and would make copying HAROS installations between machines (or creating a standard set of packages) a lot easier, since the .haros directory would not be used by default, and the installation within the virtualenv would not pollute the outside world.

[git-afsantos]

It is time to use proper database management. However, most use cases of HAROS should be relatively minimalistic, and so I will start with a minimalistic database manager as well.

One of the best candidates is TinyDB:

• small
• MIT license
• favours JSON storage

I am also considering splitting storage into multiple databases/files/documents per project. For instance, not all HAROS commands need access to previous metrics or issue reports, so these should be in their own storage.

Another example of independent storage is the parsed models for Nodes.

The big question is how to handle packages and files.

1. Use a project-independent database for packages and files. These exist independently of a project within a system, and there are probably no duplicates (except in the case of multiple workspaces), so it could be advantageous to store them only once. The downside is that all packages and files would have to be loaded in order to get the packages specific to a project.

2. Use a package database per project, containing only the packages and files relative to that project. This is closer to the current way data is stored, and easily supports multiple workspaces and package versions. It is also lighter on disk I/O when loading data. The downside is the potential for data duplication, when one package belongs to multiple projects.

[git-afsantos]

Looking at the most recent use cases of HAROS, I am also considering a few changes to the analysis targets. Namely, I think the new version should allow

• individual file analysis,
• individual package analysis,
• workspace analysis,
• individual model analysis, and
• project (custom grouping, like the current version) analysis.

Although the project concept makes sense in some cases, I am thinking how it could be improved or simplified. Right now, there is too much stuff going into this file, unnecessarily.

[git-afsantos]

I am also considering moving the configuration/node specifications from the project YAML files into their own files. Then, the project YAML files could simply reference which configs/nodes to include in the analysis, much like it already does for packages.