Store goal/constraint/condition jars in database
Rework the constraint, goal, and scheduling condition tables to accept jars.
Discussed this with @Mythicaeda @JoelCourtney @skovati and @mattdailis this morning - thanks all! We agreed on some driving requirements as well as a proposed implementation in terms of DB changes:
Requirements
Generally, we need to support procedural scheduling, which will allow users to add a new type of goal - a procedural goal with an uploaded JAR file.
- Once these procedural (JAR) goals are created, users should be able to add them to their model/plan goal specs in the same way they do (or will) for EDSL goals.
- It should be possible to have both EDSL goals and JAR goals in a spec, and even "interleave" them in terms of priority.
- Procedural goals should use the same versioning mechanism that was built for the shared EDSL goals, and it should be possible to create a new version of an existing procedural goal with a newly-uploaded JAR file.
- Procedural goals will allow arguments/parameters to be passed to them, so the DB (and UI) will need to be updated to persist and show them. This is a new feature not currently supported by EDSL goals. ~~5. Goal specs should be allowed to have the same goal multiple times, with different sets of arguments. Specs should be thought of as a list of goal instances or invocations with their own IDs, as opposed to the current design where a spec refers to goal IDs~~ moved to #1449
- Existing scheduling analysis features should support analysis of procedural goals & their parameters.
- The scheduling analysis table should not depend on the spec or any goal invocations within it, and should always reflect the state of goals at the time goals were run, even if they have changed since then. This is different from the current implementation, where if you remove a goal, it is also removed from analysis.
Proposed DB Changes
These are the proposed changes we agreed on in the meeting, @Mythicaeda please feel free to edit this list if I missed anything or if more changes are needed to satisfy the requirements above.
- ~New upload endpoint, specifically for uploading a procedural goal definition/JAR~ see #1303
- New addition to goal spec table(s): arguments field (JSONB type), representing parameters passed to the goal
- New primary key on goal spec table for representing goal invocations - either a synthetic key or a new column
- Changes to goal definition table:
- Add new field:
languagewhich can be either "EDSL" or "JAR", to represent which type it is. (naming open to discussion) - Existing
definitionfield may now either represent the EDSL source code, OR a pointer to a JAR, depending on value of language column
- Add new field:
- Changes to goal analysis tables:
- new fields: arguments (JSONB) and priority (integer)
- PK on these tables needs to change - either synthetic key of 4 columns, or new ID field
- there are three goal analysis tables that all need this change
The current plan is to have @Mythicaeda work on this after the DB refactor/merge (in progress), but we can discuss if it makes sense to have someone else help with this.
Goal specs should be allowed to have the same goal multiple times, with different sets of arguments. Specs should be thought of as a list of goal instances or invocations with their own IDs, as opposed to the current design where a spec refers to goal IDs.
In our first cut - we added arguments, without relaxing the uniqueness constraint on goal id.
Work to go: design for how to allow for multiple invocations of the same goal in the same spec
Hooked up DB changes to scheduler-worker flow.