Create internal update system

[devinhalladay]

trafficstars

We need an internal update system to keep everyone up-to-date with the latest version of Spotio.

I have absolutely zero experience with anything like that so I'd appreciate all the help I can get!!

The ideal functionality would be something like this:

• When Spotio is launched, check to see if an update has been posted on Spotify's server.
• If it has, prevent the default app's update and alert me that there's an update
• I'll update the Spotio code and push a new release
• Then the Spotio app will download the latest release

Again, I have NO idea how this kind of thing works so I'm just spitballing, but this feature is a MUST HAVE. Anyone up for the challenge??

[avelis]

Not an expert in this either but have seen some suggestions to a feature for this on wmail

Some comments made suggestions to:

1. sparkle-project
2. electron-updater

The overall feature was not build due to prohibitive costs to implementing a update/release system & signing the application for any given OS implementation. But Spotio might have different needs.

Hope this helps.

[sandervankasteel]

Last night, I had a quick look for the usual suspects in the Javascript files inside the OS X app. But I was unable to find any references to an auto-updater in the Javascript. Ideally that would be the best place to hook in a Spotio updater.

So my current guess is, that the update functionality is baked inside of the binary. :(

I hope there are some Javascript API calls available in CEF (Chromium Embedded Framework), that trigger an event (or a Javascript callback) when an update is available for downloading. I will look into this and report back.

[devinhalladay]

Thanks for the help @avelis and @sandervankasteel!! Yeah I'm assuming they're doing updates via the binary, so it might not be possible to stop them entirely from within the app.

I think a good approach might be to create an external update helper (maybe with Sparkle or electron? I don't have any experience with either but I can look into them) that blocks outgoing connections to spotify's update server and handles updates on its own whenever it sees an update available on the server

[devinhalladay]

I'm not sure how that'd work logistically yet but in theory it'd be possible. I'm also very worried about how to diff Spotio's assets with Spotify's assets when an update occurs since Spotio's asset codebase is beginning to diverge from Spotify's default assets. Anyone have thoughts on this?

[pthm]

Bit of a shot in the dark here.

Would it not be possible to wrap the Spotify binary in an electron app that would check for new versions with a checksum and apply the needed skinning?

Then you could auto update changes to the skin and make sure that Spotify is always running on the latest version.

[sandervankasteel]

My "attack" plan on this is based around a theory.

The rough theory : I doubt that Spotify will send a complete new binary DMG file to the client when an update is available. My guess is that they let the client fetch a zip / tar.gz / gzip / bz2 file with only the changes based on their current version. If that the case, they will probably have a patch / replace function in their big binary blob that handles dealing with that zip / tar.gz / gzip / bz2 file and checksumming any files.

If my theory is right; that means the Spotify.app directory (atleast on OS X) will never be completely replaced and that means we could theoretically(!) inject / load our own Javascript, which could handle our own internal updating.

My tactic on the logistics of this, use GitHub releases and let our 'versions' match the versions of Spotify. So the client does a call to Github asking for the last release, checks if the latest version on Github matches the client's local version and applies patches locally if necessary.

Github API call for getting releases : https://developer.github.com/v3/repos/releases/

But if I am wrong ..... I need some more time to think about a solution.

@pthm regarding your idea.. I like your idea, but I'm hesitant in using a solution like that because it adds an extra wrapper around an already kind of bloated application.

[sandervankasteel]

Addition to my previous theory: this doesn't apply to to Linux, since it relies on the package manager of the local machine. The situation is unknown to me on Windows.

What is also an option is that we turn Spotio into a standalone patcher application using Electron or CEF and logistically we could still use the same method I described earlier with matching our versions with the Spotify versions. And figure out that way if we need to patch the local Spotify installation.

[sandervankasteel]

I just got home started Spotify and was able to see this screen;

From that I can only conclude that Spotify does some internal checksumming to check of the files have been tampered with. This is something we should take into account.

[devinhalladay]

I'm thinking that that error is caused by having multiple versions of the Spotify app signed in on your computer. Not positive. I'm going to try to recreate their .spa file format (I believe it's just a .zip renamed to .spa, but when I tried doing that it didn't work correctly)

[devinhalladay]

@sandervankasteel I think your attack plan sounds good and logistically possible, we'll have to play around with some code and see if we can hack this together as an MVP before we can move forward with a release version though.

Another feasible (?) option would be to inject a helper that replaces all text matching a string in the updated assets with Spotio's own local strings (i.e. #333333 replaced with #ffffff, etc.). This would pose the issues that: there will be no fine controls on our end to control the visual updates; if we change one instance of the string, we will likely be changing them all; it may be very unpredictable as a process since spotify could add rules to the css that we did not foresee while developing the update system and this will break the app's design.

I'd ultimately like to pick the version of the update system that is most usable by everyday Spotify listeners (i.e. non-designer/developers), but if we must sacrifice a subset of users in order to satisfy a larger portion, I'm okay with that.

[devinhalladay]

Sorry if my thought process wasn't clear in that last comment; it's late and I'm exhausted. I'll clarify if anyone needs it.

[sandervankasteel]

I wanna see if I can build a MVP of an Electron / CEF patcher application, this weekend. It's definitely gonna be "quick 'n dirty", barely functioning and will most likely only support Linux and Mac OS for the moment. But should give a clear direction if it feasible to go into this direction.

If it isn't feasible, then we know we should be redistributing Spofity's binaries.

[devinhalladay]

Thanks, go for it! Just a basic proof of concept would be cool and if it's possible we can run with it and I can work on the Windows support and clean it all up based on your MVP. I wish I could put more time into the project right now but I'm super busy with school

[sandervankasteel]

@devinhalladay Right now my school hasn't started yet, so I have some spare time left! Once my school starts I will have the same problem ;)

[devinhalladay]

So, I think scripting it would be the best option (springing off of the conversation in #23)—my thought currently is that this is what we need:

• helper app (cross-platform, obviously), to block connections to Spotify's update servers
• an option (developer mode, not sure how this would work yet) so that I can bypass the connection block and test new updates on my machine
• this helper app will also act as an updater
• so, when I push a new release, the helper app will work relatively the same as any other update system: it will pull down the zip, replace the files in your local spotify app with the updated files
• relaunch the spotify app, and then keep listening for new updates

Since this needs to be cross-platform, I'm thinking we need to find a way to work with Electron, unless we can find developers who are willing to contribute each system's update app individually according to some code standards to keep them unified.

[devinhalladay]

Any thoughts on that? It's the only way I can think of doing this pragmatically—we need to block Spotify's update servers, one way or another, in order to keep our updates flowing smoothly, and we also need to have our own streaming update system in place. This will likely mean that I need to restructure the build system in order to more readily make updated assets (.spa's) available to the helper app so that it can simply unzip the release to a temporary directory, copy over the files to the appropriate Apps directory, then delete the temp dir. That's the easy part, though.

[devinhalladay]

Looks like we can probably just use this, but Linux will have to fall by the wayside https://medium.com/@svilen/auto-updating-apps-for-windows-and-osx-using-electron-the-complete-guide-4aa7a50b904c#.929r70t30

[devinhalladay]

Starting work on this now, using Electron.