custom repository plugings

[quasarea]

would be great to be able to have repository handlers exposed, for now you have git + svn + file, but in some cases I use tfs or clearcase or mercurial, all have some commandline support, so maybe it would be doable to have some api or configurable plugs

[boyter]

So this is somewhat currently already there using template methods, see the following directory,

https://github.com/boyter/searchcode-server/tree/2dbae0202f800a21705ce25ab4dacce1525b2dc0/src/main/java/com/searchcode/app/jobs/repository

The classes in there extend the following,

https://github.com/boyter/searchcode-server/blob/2dbae0202f800a21705ce25ab4dacce1525b2dc0/src/main/java/com/searchcode/app/jobs/repository/IndexBaseRepoJob.java

And override the various methods to implement this. That said it currently does require a rebuild of the application itself. It is the best way currently though.

I had considered as you say a plugin to do this but I think maybe a customisable way of creating command line outputs might be a better solution. The only things that need to be implemented from memory are the following,

• updateExistingRepository - given the locations needs to update the repository EG git pull, svn update etc...
• getNewRepository - given the locations should clone/checkout a new version of the repository EG git clone, svn checkout
• ignoreFile - Which special directories should be ignored EG .git .svn .hg

There would be some additional things required I would suspect though. Since SVN already uses command line it would be a good one to try and move over to this.

[quasarea]

Any luck with kicking this off?

[boyter]

Have not looked at it too seriously at the moment as its not a bug and lower on the list of priorities. I am in the middle of refactoring a lot of the logic in this area though so its something I will be working on in the future, possibly after I get the license stuff done in #96

[boyter]

So I started looking at this. For the initial version I am thinking that delta updates will not be supported. This means that the indexer will process files that have not changed between updates. In effect a reindex without pulling down everything again.

Below is a sample config that would work for SVN based on the native version. Anything between % would be replaced at runtime with the required value and all commands would run in the root directory. Probably enough here to start with.

The main one would be getting the owner of the file. This could either be hardcoded such as File Repo or use the command line to work out the owner, which should return as a simple string, or perhaps include a regex which can be used to pull back what it should be as the first capture group or something.

Also need to include branches in the below as that is something else that can be passed in

%BRANCH% might work as a replacement string.

[
    {
        "name": "SVN",
        "ignore_end_with": ["/.svn"],
        "ignore_contains": ["/.svn/"],
        "binary_path": "/usr/bin/svn",
        "update": ["%BINARY%", "update"],
        "update_with_password": ["%BINARY%", "update", "--username", "%USERNAME%", "--password", "%PASSWORD%"],
        "clone": ["%BINARY%", "checkout", "--no-auth-cache", "--non-interactive", "%REMOTELOCATION%", "%REPONAME%"],
        "clone_with_password": ["%BINARY%", "checkout", "--no-auth-cache", "--non-interactive",  "--username", "%USERNAME%", "--password", "%PASSWORD%", "%REMOTELOCATION%", "%REPONAME%"],
        "owner": ["%BINARY%", "info"]
    }
]

[quasarea]

Looks good, is that available in any branch?

[boyter]

Currently no. Still very much in development. Was mainly posting here to let you know that I am working on it.

[quasarea]

great, thanks, can't wait to see the progress :)

[boyter]

Going to push this out to the next release 1.3.12 because the current one is taking too long due to the indexing changes being larger than anticipated.

[boyter]

This is now high priority as it is a requirement for searchcode.com to be moved over to the codebase.