Backwards compatibility

[funwithtriangles]

As the app changes, project files can suddenly break everything if certain items are missing specific properties. At some point we'll want backwards compatibility so that this doesn't happen from version to version. One easy fix is for missing top level "core nodes" (e.g. A/B mix value) to be added if missing from a project. This has been discussed over at PR https://github.com/nudibranchrecords/hedron/pull/183.

There are other issues involving deeper changes to node structures that would need a little more traversing to fix. One solution would be to save the version number of the app into each project and check against it on load. If the version number doesn't match, an upgrade procedure is carried out to add any missing parts to certain nodes. This idea is nice but could be a lot of work to maintain, with quite a bit of room for error.

I think at this stage in the process it might make sense to keep backwards compatibility to a minimum and accept that old projects are going to break, until we're closer to version 1 and have some more of the core features in place. Hopefully by then the breaking changes from version to version might be a little less drastic.

[funwithtriangles]

After some thought, I can see the advantages of having project upgrade functionality at these early stages of the app.

The strategy of updating the core nodes (https://github.com/nudibranchrecords/hedron/pull/183#issuecomment-465558042) may not be necessary if we're writing version update functions for each version, as this piece of the state can be added here instead. What's nice about this is that it's very easy to see how the state structure changes from version to version. Here's my proposed method:

• Projects store the current Hedron version when saving
• If an old project is loaded into a newer version of Hedron, a warning appears for the user. A backup of their project is automatically saved (e.g. myproject.backup-v0.4.1.json) and if the user proceeds, the project is updated.
• The update procedure will be a series of functions. Every time a new version of Hedron is released, there needs to be an update function that comes with it. Simple example of what this might look like:

// upgraders/0.5.0.js
export default project => {
  // Add sketch organisation core node if it doesnt exist
  if (project.nodes.sketchOrganization === undefined) {
    project.nodes.sketchOrganization = {
      title: 'Sketch Organization',
      type: 'select',
      value: 'category',
      id: 'sketchOrganization',
      options:[
        {
          value: 'folder',
          label: 'Folder',
        },
        {
          value: 'category',
          label: 'Category',
        },
        {
          value: 'author',
          label: 'Author',
        },
      ],
    }
  }

  return project
}

• We'll need to loop through multiple of these updater functions if the version is more than one old. We'd need a way to loop through these functions, passing the previous state into the next one. Possibly will want to use a set here.
• If a version is missing in this set (e.g. the version doesn't affect project structure) then we'll need to make sure it doesn't break things if the project's version is missing in this set. One way would be to loop through the set and start looping once the project version is less than the version in the set. npm-semver looks useful here.
• Might be able to use node to our advantage here and grab all the updater functions from a directory rather than maintaining a list of them. Could order them with semver-sort.

[cale-bradbury]

I think it would be really good if we could save on code duplication, perhaps we just keep object 'chunks' somewhere and they could be referenced in multiple places as needed, this is looking like it would duplicate what exists in setCoreState

[funwithtriangles]

Yep, it definitely might start causing a lot of duplication. I don't think we need to keep the chunks anywhere special, we can just export various things from the modules where they are being used, so they can be imported into the upgrade functions as required. Of course this could make the main codebase a bit messier.

One thing that makes me think we might want duplication is that if the chunks change over time, there's a small chance it might break the upgrading functionality as it may be expecting one structure and getting another. Perhaps it's fine for these duplicate snippets of code to be "locked" in time, even if it's not very DRY. See what I mean? I'm okay with this part of the codebase to feel a bit messy as I feel this job is always going to be a bit dirty like that...

[funwithtriangles]

For now, my feeling is that we'll be adding a lot of extra work for ourselves in these early stages of development and so I don't really want to go down that route until v1. Especially as there aren't a whole bunch of users yet.

However, one solution for now would be to make a separate repo containing simple node scripts that converts save files. They could be named "0.5-0.6", "0.6-0.7", etc. This way, we can make fixes for ourselves as we need them but aren't opening the scope of the software at this early stage.

[cale-bradbury]

Could we have these files in the main hedron repo? Makes sense to include anything needed for upgrading in a PR that would require it

[funwithtriangles]

Sure, we could have a folder outside of src. I think this is a good starting point to get a feel for how backwards compatibility will work in the main app.

[funwithtriangles]

I've started on this now: https://github.com/nudibranchrecords/hedron/tree/dev/projectFixScripts