grunt-deployments icon indicating copy to clipboard operation
grunt-deployments copied to clipboard

Integrate 3rd party Search Replace DB script

Open alkah3st opened this issue 12 years ago • 29 comments

Does the find/replace part of this task account for serialized data?

alkah3st avatar Sep 04 '13 14:09 alkah3st

Not at the moment. I'm planning on making use of a 3rd party script to integrate this. However pull requests are more than welcome.

getdave avatar Sep 20 '13 09:09 getdave

@alkah3st I've recently upped by knowledge of NPM. I now understand we can reference a Git repo as a dependency. Therefore I propose the following

  1. Define https://github.com/interconnectit/Search-Replace-DB as a dependency in package.json
  2. Replace the sed (or allow as fallback?) with searchreplacedb2cli.php CLI script to account for Serialized data. We can use exec to do this easily.

If you're experienced in Grunt/Node I'd welcome some help on this.

getdave avatar Nov 12 '13 14:11 getdave

Sweet, searchreplacedb is exactly the script I use to do the find/replace. I'm not by any means a pro with Grunt/node but I'll see what I can do.

alkah3st avatar Nov 12 '13 15:11 alkah3st

Should be pretty simple. I'm planning to get to this Thurs/Fri

getdave avatar Nov 18 '13 14:11 getdave

@alkah3st if you're interested I've made some good progress on this at

https://github.com/getdave/grunt-deployments/tree/feature/advanced-search-replace

Requires a lot of testing however. Would you be able to give it a whirl and see if it works for you?

getdave avatar Jan 03 '14 16:01 getdave

Awesome, I'll check it out this weekend. I just in fact was retooling my own website, so I can experiment on it there.

alkah3st avatar Jan 03 '14 18:01 alkah3st

That would be much appreciated. Be sure to do a fresh install. Dependencies have changed a lot. NPM update...

Cheers

getdave avatar Jan 03 '14 19:01 getdave

@alkah3st Just checking to see if you've had any chance to test this out?

getdave avatar Jan 17 '14 12:01 getdave

Hi Dave, sorry I have disappeared from the face of the earth these past two months. I've been settling into a new job. Quick question I have about your advanced search/replace deployments script: I have everything set up properly in my gruntfile, but what's not clear to me is steps 1) and 2) you described above:

  1. Define https://github.com/interconnectit/Search-Replace-DB as a dependency in package.json
  2. Replace the sed (or allow as fallback?) with searchreplacedb2cli.php CLI script to account for Serialized data. We can use exec to do this easily.

I understand I need to include a reference to searchreplacedb2 script in my package.json, but am I including the git path (i.e., https://github.com/interconnectit/Search-Replace-DB.git) in the package like so:

"grunt-deployments": "~0.2.0", "https://github.com/interconnectit/Search-Replace-DB.git": ??

Also I'm not sure what you mean by #2.

alkah3st avatar Feb 20 '14 18:02 alkah3st

@alkah3st Not a problem.

I've actually already done most of the work on this branch

https://github.com/getdave/grunt-deployments/tree/feature/advanced-search-replace

Check out this branch and give it a whirl. I'm interested to see what issues you encounter.

getdave avatar Feb 21 '14 11:02 getdave

Ah cool, that's the one I'm testing out. I have an environment where I'll regularly be moving a WP site from dev -> stage -> live, so I think this would be an ideal test.

alkah3st avatar Feb 21 '14 13:02 alkah3st

@alkah3st I would be great if you could test this. Be careful with it as it's not production-ready code yet!

getdave avatar Feb 24 '14 09:02 getdave

Here's a question for you: so I have everything set up in my external file:

{ "local": { "title": "Local", "database": "xxx", "user": "xxx", "pass": "", "host": "xxx", "url": "xxx" }, "develop": { "title": "xxx", "database": "xxx", "user": "xxx", "pass": "xxx", "host": "xxx", "url": "xxx", "ssh_user": "xxx", "ssh_host": "xxx", "ssh_port": "xxx" }, "stage": { "title": "xxx", "database": "xxx", "user": "xxx", "pass": "xxx", "host": "xxx", "url": "xxx", "ssh_user": "xxx", "ssh_host": "xxx", "ssh_port": "xxx" } }

as .dbpass, and the task in my gruntfile is:

    deployments: {
        options: {
          backups_dir: 'db'
        },
        local: '<%= untracked_targets.local %>',
        develop: '<%= untracked_targets.develop %>',
        stage: '<%= untracked_targets.stage %>'
     },

When I run db_pull --src="local" --dest="develop" I get this error:

"Warning: An error occurred while processing a template (Cannot read property 'local' of undefined). Use --force to continue."

Is "untracked_targets" correct to reference the external file with the credentials?

alkah3st avatar Feb 24 '14 16:02 alkah3st

Thanks for giving this a shot. I've tried to help below...

Is "untracked_targets" correct to reference the external file with the credentials?

You can name and reference the external file however you'd like. A good explanation of how to achieve this is found on the readme. My file is similar to the below:

{
    "local": {
        "title": "Local",
        "database": "my_db_name",        
        "user": "my_db_user",
        "pass": "my_db_pass",
        "host": "localhost",  
        "url": "www.test.local"
    },
    "develop": {
        "title": "Development Server",
        "database": "my_db_name",        
        "user": "my_db_user",
        "pass": "my_db_pass",
        "host": "localhost",      
        "url": "www.mydevurl.com",
        "ssh_user": "my_ssh_user",
        "ssh_host": "111.111.11.11"  
    }
}

...and the portion of the my Gruntfile which references it is:

grunt.initConfig({
    db_fixture: grunt.file.readJSON('test/fixtures/basic_config.json'), // the path can be to any location where you'd like to store the config file
    deployments: {
        options: {
            backups_dir: ''
        },
        local: '<%= db_fixture.local %>', // make sure you've created valid fixture DB creds
        develop: '<%= db_fixture.develop %>' // make sure you've created valid fixture DB creds
    },

    // remainder of Gruntfile continues...

When I run db_pull --src="local" --dest="develop" I get this error:

In hindsight the use of db_pull as the CLI command is stupid. I would like to rename is to db_migrate. Feel free to do this via a pull request into the advanced search replace branch (or I'll do it later!).

"Warning: An error occurred while processing a template (Cannot read property 'local' of undefined). Use --force to continue."

This is because it cannot find a reference to "local". You will need to double check that the script has a "local" target. Hopefully it's just a minor typo rather than an issue with the script.

Happy to help you get this working. Sorry it's been less than smooth so far!

getdave avatar Feb 26 '14 09:02 getdave

Awesome, I will try this out once I get to the office!

alkah3st avatar Feb 26 '14 14:02 alkah3st

@alkah3st Did you have any luck?

getdave avatar Mar 03 '14 15:03 getdave

New Requirement

Now that the original SearchReplaceDB repo has been updated to include a package.json file this project should reference the original repo as a dependency rather than my forked version. This should be updated in package.json.

getdave avatar Mar 08 '14 10:03 getdave

Hi Dave, so I was diving back into this, but now it's not clear to me how I'm supposed to include the new version. My package.json references grunt-deployments:

{
  "name": "leap-wordpress-package",
  "version": "1.0.0",
  "dependencies": {},
  "devDependencies": {
   ...
    "grunt-deployments": "0.2.0"
  }
}

I'm including a config file as dbpass.json, and this is my gruntfile:

module.exports = function(grunt) {
    grunt.initConfig({
        dbcreds: grunt.file.readJSON('dbpass.json'),
        deployments: {
            options: {
              backups_dir: 'db'
            },
            local: '<%= dbcreds.local %>',
            develop: '<%= dbcreds.develop %>',
            stage: '<%= dbcreds.stage %>'
         }
    });
};

What am I missing here?

alkah3st avatar Mar 19 '14 21:03 alkah3st

Hmmm...that's a good point. Not sure of the top of my head how you'd reference a feature branch on Github via `package.json.

Might be easier to test the new Search & Replace branch in isolation from your leap-wordpress-package? Just checkout the branch and try it out...?

Sorry if this is making things difficult for you.

getdave avatar Mar 24 '14 09:03 getdave

Following these comments with interest, am I right in thinking the Search Replace DB feature branch hasn't been merged into the master as of yet?

davemac avatar Mar 30 '14 23:03 davemac

That's where I'm confused too... Not sure how to run this independently of the master.

alkah3st avatar Mar 30 '14 23:03 alkah3st

@davemac @alkah3st Sorry guys. Appreciate this is confusing.

The Search Replace DB is on a branch at:

https://github.com/getdave/grunt-deployments/tree/feature/advanced-search-replace

I would advise that you create a clean dir on your local machine and then clone a fresh copy of the repo. You can do this using

git clone -b feature/advanced-search-replace [email protected]:getdave/grunt-deployments.git

That should do it. From there you should be able to test the functionality.

Alternatively, if you already have a project setup which makes use of Grunt Deployments then I believe you can modify your package.json to refer to a branch directly

http://stackoverflow.com/questions/16350673/depend-on-a-branch-using-a-git-url-in-a-package-json

So you'd refer to the branch feature/advanced-search-replace

getdave avatar Mar 31 '14 08:03 getdave

Also I'm consider making this an optional feature. So the idea would be

  • Plugin defaults to using Search&ReplaceDB for search/replace
  • ...or you can pass an option to the task to tell it to use sed for search/replace instead
  • ...or you can pass your own function to do the search/replace manually.

Also note this task which relates to this quite well https://github.com/getdave/grunt-deployments/issues/41

getdave avatar Mar 31 '14 09:03 getdave

Thanks Dave, my vote would certainly be to make search replace db an optional feature. That would then make things easier for users, I suppose - but more work for you!

davemac avatar Mar 31 '14 09:03 davemac

OK have tried to add the advanced search replace feature branch to package.json:

"devDependencies": { "grunt": "~0.4.2", "grunt-autoprefixer": "~0.7.2", "grunt-contrib-imagemin": "~0.6.0", "grunt-contrib-jshint": "~0.9.2", "grunt-contrib-uglify": "^0.4.0", "grunt-contrib-watch": "~0.6.1", "grunt-rsync": "~0.5.0", "matchdep": "~0.3.0", "grunt-deployments": "git://github.com/getdave/grunt-deployments.git#feature/advanced-search-replace" }

But am getting an error when I run npm update:

8397 error git fetch -a origin ([email protected]:getdave/Search-Replace-DB.git) Permission denied (publickey). 8397 error git fetch -a origin ([email protected]:getdave/Search-Replace-DB.git) fatal: Could not read from remote repository. 8397 error git fetch -a origin ([email protected]:getdave/Search-Replace-DB.git) 8397 error git fetch -a origin ([email protected]:getdave/Search-Replace-DB.git) Please make sure you have the correct access rights 8397 error git fetch -a origin ([email protected]:getdave/Search-Replace-DB.git) and the repository exists

davemac avatar May 08 '14 05:05 davemac

Worked it out, you need to delete node_modules and run npm install again.

After hours of googling, I found an explanation here http://howtonode.org/managing-module-dependencies

davemac avatar May 08 '14 06:05 davemac

@davemac Nice work sir!

getdave avatar May 09 '14 11:05 getdave

@davemac If it's possible to advise on your progress with testing this. Has it worked for you? Any major errors? Any suggested improvements. As always pull requests into the correct branch are welcome.

I'd really like to get this released so I'm really appreciative of your testing efforts!

getdave avatar May 11 '14 21:05 getdave

See issue #43 which helps with getting this feature branch installed as a dependency and also #44 which I believe is an outstanding issue.

davemac avatar May 12 '14 00:05 davemac