generator-m-ionic
generator-m-ionic copied to clipboard
update functionality for the generator
read .yo-rc.json
contents, so it works similar to reapeating yo m --skip-prompts in the same directory. Be careful since .yo-rc.json
contents get overwritten without any warning. This might lead to complications when version controlling cordova platforms and plugins via .yo-rc.json
usual procedure:
- always document changes to generator core files (readme is a good place)
- gulp tasks, gitignore, eslintrc, index.html, app.js ...
- create new folder
- setup new project (same name, bundle ids, ..)
- cp
.git/
folder to new project - copy custom files and folders if you have any (
artifacts
,doc
, ...), that definitely won't have conflicts - remove modules in
app/
andtest/karma
andtest/protractor
- copy your own modules into the respective folders (spare
.eslintrc
files and such) - replace
res/
folder - git diff
- reproduce documented changes to generator files and resolve conflicts
- git add one at a time
- esp. bower components, config.xml configuration, ...
- bower install && npm install && gulp --cordova 'prepare'
- gulp watch
- review changes, commit
possible issues:
- overwrite forgotten updates to generator files (gulp task alterations, eslintrc, ...)
- app code might break due to breaking changes of bower components
- incompatible config.xml settings, platform/plugin versions
test:
- gulp watch
- gulp karma
- gulp protractor
- gulp build
- gulp --cordova 'run'
- CI build
- gulp update - creates new project (with current and new generator version) based on
.yo-rc.json
, diff only generator project
first draft: unfortunately some weird issues with using npm programatically. Upon calling install of the second generator version npm throws this error, no matter what I do: timeout, loading npm again, ...
npm ERR! install trying to install 1.7.0 to /Users/jonathan/Projects/generator-m-ionic-demo/node_modules/generator-m-ionic
npm ERR! install but already installed versions [ '1.6.0' ]
npm i --save-dev yeoman-test
in project
gulp/update.js
'use strict';
// gulp
var gulp = require('gulp');
var options = gulp.options;
var $ = require('gulp-load-plugins')();
// node core
var npm = require('npm');
var fs = require('fs');
var path = require('path');
// other
var chalk = require('chalk');
var yeomanTest = require('yeoman-test');
var answers = require('../.yo-rc.json')['generator-m-ionic'].answers;
var help = {
loadNpm: function (cb) {
npm.load({}, function (err) {
if (err) {
console.log(chalk.red('error loading npm\n'), err);
}
else {
console.log(chalk.green('npm loaded successfully\n'));
}
cb();
});
},
npmCmd: function (command, packageName, cb) {
npm.commands[command]([packageName], function (er) {
if (er) {
console.log(chalk.red('error running: ') + command + ' ' + packageName, er);
}
else {
console.log(chalk.green('successfullly ran: ') + command + ' ' + packageName);
}
cb();
});
},
runGeneratorVersion: function (version, cb) {
this.loadNpm(function () {
this.npmCmd('install', 'generator-m-ionic@' + version, function () {
process.chdir('../');
fs.mkdirSync(help.projectDirVersion(version));
process.chdir(help.projectDirVersion(version));
var generatorBase = '../' + help.projectDir + '/node_modules/generator-m-ionic/generators/';
var ctx = yeomanTest.run(path.resolve(generatorBase + '/app'))
.withGenerators(help.getDirectoryDirs(generatorBase))
.withOptions({ // execute with options
'skip-install': true, // don't need to install deps
'skip-sdk': true // for some reason won't install cordova properly, so just leave it
})
.withPrompts(answers) // answer prompts
.on('end', function () {
process.chdir('../' + help.projectDir);
this.npmCmd('uninstall', 'generator-m-ionic', cb);
}.bind(this));
ctx.settings.tmpdir = false; // don't run in tempdir
}.bind(this));
}.bind(this));
},
projectDir: (function () {
var projectDir = process.cwd().split('/');
projectDir = projectDir[projectDir.length - 1];
return projectDir;
})(),
projectDirVersion: function (version) {
return this.projectDir + '@' + version;
},
getDirectoryDirs: function (dirPath) {
return fs.readdirSync(dirPath)
.filter(function (file) {
return fs.statSync(path.join(dirPath, file)).isDirectory();
})
.map(function (dir) {
return path.resolve(dirPath, dir);
});
}
};
// UPDATE
gulp.task('update', [], function (done) {
if (!options.from || !options.to) {
console.log(chalk.red('error: ') + 'supply proper generator versions with --from and --to');
return;
}
help.runGeneratorVersion(options.from, function () {
help.runGeneratorVersion(options.to, function () {
done();
});
});
});
Some related research:
use npm programatically http://stackoverflow.com/questions/20686244/install-programmatically-a-npm-package-providing-its-version http://stackoverflow.com/questions/15957529/can-i-install-a-npm-package-from-javascript-running-in-node-js
run generators programatically http://stackoverflow.com/questions/25233234/using-yeoman-programmatically-inside-nodejs-project
get the difference between two repos http://stackoverflow.com/questions/1968512/getting-the-difference-between-two-repositories
node cache invalidation http://stackoverflow.com/questions/9210542/node-js-require-cache-possible-to-invalidate https://nodejs.org/docs/latest/api/globals.html#globals_require_cache
Tips for Writing Portable Node.js Code https://gist.github.com/domenic/2790533 http://shapeshed.com/writing-cross-platform-node/
version 2: with child_process.exec
, creates two new folders with a fresh setup according to answers. Applying some gitmagic for updates.
'use strict';
// gulp
var gulp = require('gulp');
var options = gulp.options;
// node core
var fs = require('fs');
var path = require('path');
var exec = require('child_process').exec;
// other
var yeomanTest = require('yeoman-test');
var chalk = require('chalk');
var answers = require('../.yo-rc.json')['generator-m-ionic'].answers;
var help = {
projectDir: (function () {
var projectDir = process.cwd().split('/');
projectDir = projectDir[projectDir.length - 1];
return projectDir;
})(),
projectDirVersion: function (version) {
return this.projectDir + '@' + version;
},
getDirectoryDirs: function (dirPath) {
return fs.readdirSync(dirPath)
.filter(function (file) {
return fs.statSync(path.join(dirPath, file)).isDirectory();
})
.map(function (dir) {
return path.resolve(dirPath, dir);
});
},
installAndRunGen: function (version, cb) {
// install
this.install(version, function () {
// change into new dir
process.chdir('../');
fs.mkdirSync(this.projectDirVersion(version));
process.chdir(this.projectDirVersion(version));
console.log(chalk.green('change dir: ') + process.cwd());
// execute generator
var generatorBase = '../' + this.projectDir + '/node_modules/generator-m-ionic';
var generatorGenerators = generatorBase + '/generators';
// delete require cache to make node reload the new version
this.deleteCache();
console.log(chalk.green('running: ') + require('../' + generatorBase + '/package.json').version);
var ctx = yeomanTest.run(path.resolve(generatorGenerators + '/app'));
ctx.settings.tmpdir = false; // don't run in tempdir
ctx
.withGenerators(this.getDirectoryDirs(generatorGenerators))
.withOptions({ // execute with options
'skip-install': true, // don't need to install deps
'skip-sdk': true // for some reason won't install cordova properly, so just leave it
})
.withPrompts(answers) // answer prompts
.on('end', function () {
// git
exec('git init && git add . && git commit -m \'init\'');
// uninstall
this.uninstall(version, cb);
}.bind(this));
}.bind(this));
},
uninstall: function (version, cb) {
process.chdir('../' + this.projectDir);
console.log(chalk.green('uninstalling: ') + version + ' ' + process.cwd());
exec('npm uninstall generator-m-ionic', function (error) {
if (error) {
console.log(chalk.red('error: ') + 'uninstalling ' + version + '\n', error);
}
cb();
});
},
install: function (version, cb) {
console.log(chalk.green('installing: ') + version + ' to ' + process.cwd());
exec('npm i generator-m-ionic@' + version, function (error) {
if (error) {
console.log(chalk.red('error: ') + 'installing ' + version + '\n', error);
}
cb();
});
},
deleteCache: function () {
for (var key in require.cache) {
if (key.indexOf('node_modules/generator-m') > -1) {
delete require.cache[key];
}
}
}
};
gulp.task('update', [], function (done) {
if (!options.from || !options.to) {
console.log(chalk.red('error: ') + 'supply proper generator versions with --from and --to');
return;
}
help.installAndRunGen(options.from, function () {
help.installAndRunGen(options.to, function () {
// add remotes to from-project
process.chdir('../' + help.projectDirVersion(options.from));
var gitCmds = 'git remote add -f update ../' +
help.projectDirVersion(options.to) +
'&& git remote add -f project ../' +
help.projectDir +
'&& git remote update';
exec(gitCmds, function () {
done();
});
});
});
});
Additional git-magic use in project@<from-version>
:
git diff master remotes/update/master --stat --diff-filter=M > update-diff
git diff master remotes/project/master --stat --diff-filter=M > project-diff
diff update-diff project-diff
OR:
# merge update branch into original clean state
git checkout -b magic
git merge remotes/update/master -X theirs
# then merge the project branch
git merge remotes/project/master
# resolve conflicts
=> not the same repo, could delete .git
and copy the one from original project. Rather complicated though.
version 3: applies update on-top of current repo, overwriting files
'use strict';
// gulp
var gulp = require('gulp');
var options = gulp.options;
// node core
var fs = require('fs');
var path = require('path');
var exec = require('child_process').exec;
// other
var yeomanTest = require('yeoman-test');
var chalk = require('chalk');
var answers = require('../.yo-rc.json')['generator-m-ionic'].answers;
var help = {
projectDir: (function () {
var projectDir = process.cwd().split('/');
projectDir = projectDir[projectDir.length - 1];
return projectDir;
})(),
projectDirVersion: function (version) {
return this.projectDir + '@' + version;
},
getDirectoryDirs: function (dirPath) {
return fs.readdirSync(dirPath)
.filter(function (file) {
return fs.statSync(path.join(dirPath, file)).isDirectory();
})
.map(function (dir) {
return path.resolve(dirPath, dir);
});
},
updateWithYo: function (version, cb) {
this.install(version, function () {
// execute generator
var generatorBase = '../' + this.projectDir + '/node_modules/generator-m-ionic';
var generatorGenerators = generatorBase + '/generators';
console.log(chalk.green('running: ') + require('../' + generatorBase + '/package.json').version);
var ctx = yeomanTest.run(path.resolve(generatorGenerators + '/app'));
ctx.settings.tmpdir = false; // don't run in tempdir
ctx
.withGenerators(this.getDirectoryDirs(generatorGenerators))
.withOptions({ // execute with options
'skip-install': true, // don't need to install deps
'skip-sdk': true, // for some reason won't install cordova properly, so just leave it
'force': true
})
.withPrompts(answers) // answer prompts
.on('end', function () {
// uninstall
this.uninstall(version, cb);
}.bind(this));
}.bind(this));
},
uninstall: function (version, cb) {
process.chdir('../' + this.projectDir);
console.log(chalk.green('uninstalling: ') + version + ' ' + process.cwd());
exec('npm uninstall generator-m-ionic', function (error) {
if (error) {
console.log(chalk.red('error: ') + 'uninstalling ' + version + '\n', error);
}
cb();
});
},
install: function (version, cb) {
console.log(chalk.green('installing: ') + version + ' to ' + process.cwd());
exec('npm i generator-m-ionic@' + version, function (error) {
if (error) {
console.log(chalk.red('error: ') + 'installing ' + version + '\n', error);
}
cb();
});
}
};
gulp.task('update', [], function (done) {
if (!options.to) {
console.log(chalk.red('error: ') + 'supply proper generator version with --to');
return;
}
help.updateWithYo(options.to, function () {
done();
});
});
@MathiasTim, @lordgreg, @DrMabuse23 and team. I invested some (quite a lot of) time into this and found out (as expected) that this is a very delicate and complex task.
However I have an initial proposal that I marked as experimental
. Please let me know what you think and if this is similar to what you had in mind and if you have any ideas on how to improve on this.
Before this goes live tomorrow you can find a guide here: https://github.com/mwaylabs/generator-m-ionic/blob/dev/docs/guides/generator_update.md
@gruppjo I will probably have to wait for upcoming update (1.8.0). What I've did is:
- cloned a project with generator-m 1.3.3.
- installed yeoman-test with
npm i yeoman-test --save-dev
- created new branch
git checkout -b update
- copied
update.js
from gulp directory of dev branch of generator-m and pasted it into my current project folder. - ran eperimental-update task with version 1.7.0 since 1.8.0 isn't available yet.
Here's what I get:
Gregor@HomePC /c/Development/genm-upgrade (update)
$ gulp experimental-update --to=1.7.0
[20:00:17] Using gulpfile C:\Development\genm-upgrade\gulpfile.js
[20:00:17] Starting 'experimental-update'...
installing: 1.7.0 to C:\Development\genm-upgrade
module.js:341
throw err;
^
Error: Cannot find module '../../C:\Development\genm-upgrade/node_modules/generator-m-ionic/package.json'
at Function.Module._resolveFilename (module.js:339:15)
at Function.Module._load (module.js:290:25)
at Module.require (module.js:367:17)
at require (internal/module.js:16:19)
at Object.<anonymous> (C:\Development\genm-upgrade\gulp\update.js:41:46)
at C:\Development\genm-upgrade\gulp\update.js:77:7
at ChildProcess.exithandler (child_process.js:193:7)
at emitTwo (events.js:100:13)
at ChildProcess.emit (events.js:185:7)
at maybeClose (internal/child_process.js:850:16)
at Socket.<anonymous> (internal/child_process.js:323:11)
at emitOne (events.js:90:13)
at Socket.emit (events.js:182:7)
at Pipe._onclose (net.js:477:12)
That was, however, after waiting approximately 30 seconds in the terminal without knowing if there's any progress or not.
$ npm --version
3.7.3
Gregor@HomePC /c/Development/genm-upgrade (update)
$ node --version
v5.9.1
So what about migrations? Maybe with a diff From your git repo from Tag you can find out which files are habe to be changed. And maybe we can add a migration json Form Every Version to Version its simple but strong
Sent from my Cyanogen phone
Am 28.04.2016 8:06 nachm. schrieb Gregor [email protected]:
@gruppjo I will probably have to wait for upcoming update (1.8.0). What I've did is:
cloned a project with generator-m 1.3.3.installed yeoman-test with npm i yeoman-test --save-dev created new branch git checkout -b update copied update.js from gulp directory of dev branch of generator-m and pasted it into my current project folder. ran eperimental-update task with version 1.7.0 since 1.8.0 isn't available yet.
Here's what I get:
Gregor@HomePC /c/Development/genm-upgrade (update) $ gulp experimental-update --to=1.7.0 [20:00:17] Using gulpfile C:\Development\genm-upgrade\gulpfile.js [20:00:17] Starting 'experimental-update'... installing: 1.7.0 to C:\Development\genm-upgrade module.js:341 throw err; ^ Error: Cannot find module '../../C:\Development\genm-upgrade/node_modules/generator-m-ionic/package.json' at Function.Module._resolveFilename (module.js:339:15) at Function.Module._load (module.js:290:25) at Module.require (module.js:367:17) at require (internal/module.js:16:19) at Object.
That was, however, after waiting approximately 30 seconds in the terminal without knowing if there's any progress or not.
$ npm --version 3.7.3 Gregor@HomePC /c/Development/genm-upgrade (update) $ node --version v5.9.1
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub
@lordgreg, please use at least npm 3.8.3 or higher. This version fixed some of the performance issues of npm.
@lordgreg, it seems like the paths are not windows-safe. Wops. I'll try to fix that. The wait is normal, since the installation takes a while and I don't output the buffer, so you don't see the usual installation progress. Which I'd like to have in a future version but I'm not familiar with buffers and streams in JS so much and I really want to release this version soon.
@DrMabuse23, I like the idea of providing a diff. We could do that in every release for the previous version and mention in the doc that you can create the diffs between the versions you need yourself. I'm not sure what you mean by migration json?
@lordgreg, please try the new version of update.js. It should now be windows safe!
@gruppjo will do roger wilco (aka today evening). Will report my result afterwards. 👍
I will leave this issue open so we can improve on this in future versions together.
So, here's how my update just went. No errors:
Gregor@HomePC /c/Development/genm-upgrade (update)
$ gulp experimental-update --to=1.8.0
[19:13:14] Using gulpfile C:\Development\genm-upgrade\gulpfile.js
[19:13:14] Starting 'experimental-update'...
installing: 1.8.0 to C:\Development\genm-upgrade
running: 1.8.0
Gregor@HomePC /c/Development/genm-upgrade (update)
$ git status
On branch update
Untracked files:
(use "git add <file>..." to include in what will be committed)
gulp/update.js
jsconfig.json
nothing added to commit but untracked files present (use "git add" to track)
Gregor@HomePC /c/Development/genm-upgrade (update)
$ git log
commit e7d3ef57c07514bc6fa7be375962c97b82fa7661
Author: Gregor <[email protected]>
Date: Thu Apr 28 19:54:58 2016 +0200
initial commit
Checking README.md, there's still version 1.3.3 listed. diff shows no other things were updated, which leads me thinking... did it worked? :8ball:
@lordgreg, it most likely did not. The output is not complete. This is how it should look:
$ gulp experimental-update --to=1.8.0
[15:45:30] Using gulpfile ~/Projects/generator-m-ionic-demo/gulpfile.js
[15:45:30] Starting 'experimental-update'...
installing: 1.8.0 to /Users/jonathan/Projects/generator-m-ionic-demo
running: 1.8.0
uninstalling: 1.8.0 /Users/jonathan/Projects/generator-m-ionic-demo
[15:45:59] Finished 'experimental-update' after 29 s
You're missing the second half.
I'm a little confused. It seems like the task is failing but silently. It's very inconvenient that I cannot test this on windows myself. You could insert some console.logs
to see where it fails. It appears to fail somewhere along these lines. https://github.com/mwaylabs/generator-m-ionic/blob/master/generators/app/templates/gulp/update.js#L32-L50
Addy Osmani over at Yeoman, mentions the "emit" feature that has been coming out of the CLI ecosystems. https://github.com/yeoman/yeoman/issues/1265#issuecomment-285806051
Removed
As part of #481, we're removing this feature from the master.
Alternatives
1 Manually upgrade
You can manually upgrade the generator by creating a new project and moving your old files to the new project. A good step by step guide I have written down here for your consideration: https://github.com/mwaylabs/generator-m-ionic/issues/158#issuecomment-168699072
2 Run yo m-ionic
When running yo m-ionic
again in your project's directory and it will run the generator again, using your old answers (except for a few ones that you will prompted to answer again). Then use yeoman's diff feature to look for changes.
3 CLI and "emit"
Maybe the future of generators is a CLI/Generator hybrid that can emit the config upon request as mentioned above.
Addy Osmani over at Yeoman, mentions the "emit" feature that has been coming out of the CLI ecosystems. https://github.com/yeoman/yeoman/issues/1265#issuecomment-285806051
The current implementation for reference
'use strict';
// gulp
var gulp = require('gulp');
var options = gulp.options;
// node core
var fs = require('fs');
var path = require('path');
var exec = require('child_process').exec;
// other
var yeomanTest = require('yeoman-test');
var chalk = require('chalk');
var answers = require('../.yo-rc.json')['generator-m-ionic'].answers;
var help = {
getDirectoryDirs: function (dirPath) {
return fs.readdirSync(dirPath)
.filter(function (file) {
return fs.statSync(path.join(dirPath, file)).isDirectory();
})
.map(function (dir) {
return path.resolve(dirPath, dir);
});
},
updateWithYo: function (version, cb) {
this.install(version, function () {
// execute generator
var generatorBase = path.resolve('node_modules/generator-m-ionic');
var generatorGenerators = path.join(generatorBase, 'generators');
console.log(chalk.green('running: ') + require(path.join(generatorBase, '/package.json')).version);
var ctx = yeomanTest.run(path.join(generatorGenerators, '/app'));
ctx.settings.tmpdir = false; // don't run in tempdir
ctx
.withGenerators(this.getDirectoryDirs(generatorGenerators))
.withOptions({ // execute with options
'skip-install': true, // don't need to install deps
'skip-sdk': true, // for some reason won't install cordova properly, so just leave it
'force': true
})
.withPrompts(answers) // answer prompts
.on('end', function () {
// uninstall
this.uninstall(version, cb);
}.bind(this));
}.bind(this));
},
uninstall: function (version, cb) {
console.log(chalk.green('uninstalling: ') + version + ' ' + process.cwd());
exec('npm uninstall generator-m-ionic', function (error) {
if (error) {
console.log(chalk.red('error: ') + 'uninstalling ' + version + '\n', error);
}
cb();
});
},
install: function (version, cb) {
console.log(chalk.green('installing: ') + version + ' to ' + process.cwd());
exec('npm i generator-m-ionic@' + version, function (error) {
if (error) {
console.log(chalk.red('error: ') + 'installing ' + version + '\n', error);
}
cb();
});
}
};
gulp.task('experimental-update', [], function (done) {
if (!options.to) {
console.log(chalk.red('error: ') + 'supply proper generator version with --to');
return;
}
help.updateWithYo(options.to, function () {
done();
});
});
The current docs:
Generator Update (experimental)
This feature is highly experimental!
We're exploring ways that can make upgrading your project to a new version of the generator easier. We're very happy to get feedback from you, but please be careful with what you are doing. The full discussion with many different ideas on how to achieve an update can be found in this issue.
Considerations
When you are trying to upgrade your project there's many things to consider that stem from a complex technology stack, how you work on your project, whether and how you use git and many other factors. So depending on these topics there might be better ways of how to perform an update. Please feel free to explore them and share them with us.
Additional considerations that play a role on how easy an update is:
- What version are you upgrading from (that's the version of the generator you used when you called
yo m-ionic
for your project the first time)? - Which version are you upgrading to (most likely the latest one)
-
What changes were introduced to the generator in between those two versions?
- only gulp file update, new features and subgenerators?
- or possibly larger, less isolated changes that affect many aspects of the generator?
- Did you change any of the generator's core files after project setup or just add new stuff?
- gulp tasks, config files like
.eslintrc
and theindex.html
would be core files for instance. Basically everything that is generated on the first run ofyo m-ionic
- gulp tasks, config files like
Additional/alternative measures
- Consulting the Changelog and the changes between the two version in the generator repository and especially the demo repository can be enormously helpful.
- You might want to simply create a new project with
yo m-ionic
and piece by piece copy your stuff
Versions <1.8.0
If you're upgrading from an version of the generator that is smaller than 1.8.0. You'll need to do two things:
# install yeoman-test as a devDependency (in your project folder)
npm i yeoman-test --save-dev
And save the following gulp file as gulp/update.js
in your project.
Perform experimental update
First of all we recommend switching to a new git branch (just to be sure).
git checkout -b update
Before you perform the next step.
Read this carefully. This will do the following:
- temporarily install the new generator version locally
- take the contents of your
.yo-rc.json
(the choices you made when runningyo m-ionic
to set up your project) - pass these answers into the new generator version
- run the generator on top of your current project
- overwrite all files that the generator generated in the first place with the new ones
- thus very very likely break your code
=> It's up to you to make sense of these changes and differentiate between changes we made and possible changes you made, and reapply them when necessary.
This will not:
- delete any files that became obsolete with the new version (doesn't happen often)
- take into consideration any of the changes you made to the core files (how would we know?). They will be overwritten.
- manipulate any other files, and magically make your code work
- re-run any sub-generators you used
There's still some things that could go wrong:
- your
.yo-rc.json
is missing some answers (because they weren't available when you set up the project). To remedy this, just add the desired values for those answers.
If you are lucky and additionally:
- the version you're updating form is not too old
- you didn't drastically change any of the generator's core files
- you're smart, used git properly, know your project and all that
- then ...
... updating might be very simple.
No promises!
Then, after careful consideration of all of this, run:
# update to version 1.8.0
gulp experimental-update --to=1.8.0
# wait
git diff
Good luck and please give some feedback in this issue!