Add repositories quickly without full clone (was: can't add private Gitosis repos)

[danlynn]

I use gitosis to host all my private repos (http://blog.ardes.com/2008/5/19/git-hosting-with-leopard, http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way, http://eagain.net/gitweb/?p=gitosis.git). It uses regular ssh protocol with standard public key - just like github. When I attempt to add any of my repos, the add dialog simply sits and spins forever. It is a little puzzling - ssh with public key works with github - but not with the repos on my own domain hosted via gitosis.

[mackuba]

And if you clone the same repository via 'git clone' in the command line, it just clones it without asking about anything?

[danlynn]

Yes. It is 100% standard ssh git cloning with a key pair.

$ git clone [email protected]:SearchClient Initialized empty Git repository in /Users/danlynn/Documents/temp/deleteme/SearchClient/.git/ remote: Counting objects: 75, done. remote: Compressing objects: 100% (58/58), done. remote: Total 75 (delta 23), reused 0 (delta 0) Receiving objects: 100% (75/75), 24.52 KiB, done. Resolving deltas: 100% (23/23), done. $

Actually, it appears to be working. I tried a tiny repo (like the one above) and it worked. Perhaps the bigger repo's that I tried to add earlier were simply taking a long time to clone. Perhaps it would be a good idea to display a line or two of status scrolling in the add dialog showing the console output as it is transferring objects (only if not completed after 5 seconds). This would prevent people like me from thinking that it is hung.

[mackuba]

I see... I thought the "please be patient" message would be enough, but you're right that if it takes too long, you start wondering if it really is doing anything. I'll see if I can show some more info...

[danlynn]

It's not obvious that it is actually cloning a repo and that it may take a while. The dialog is simple enough that it appears that it is doing nothing more than simply adding a config line. Perhaps the cloning should be done in the background and the list of repos should have an icon next to each showing that it is currently cloning, cloned, checking. This way the dialog is really just a config entry and not actually performing work. At the same time, the list of repos will give better status info on each.

[mackuba]

Well, that's exactly why there is a spinner and a message "Please be patient, I'm cloning the repository" - so that you don't think it's just adding a config line :) I don't want to run the cloning asynchronously, because until the clone command returns, I have no idea if you've entered a correct URL - maybe you made a typo, maybe you used wrong protocol, or something. I will add some more info to the dialog though - either a progress bar, or a line displaying the output from git command.

[danlynn]

Try this to quickly verify that the URL is good:

$ mkdir test_git
$ cd test_git/
$ git init
Initialized empty Git repository in /Users/danlynn/Documents/temp/delete/test_git/.git/
$ git remote add origin [email protected]:POLocatorClient
$ git remote show origin
* remote origin
  URL: [email protected]:POLocatorClient
  New remote branches (next fetch will store in remotes/origin)
    danlynn_dev
    master
    subversion
    try_atg_2
    try_atg_jars
    try_multiprotocol
    waipang_dev
$ 

This is what it looks like when the URL is bad:

$ mkdir test_git_bad
$ cd test_git_bad/
$ git init
Initialized empty Git repository in /Users/danlynn/Documents/temp/delete/test_git_bad/.git/
$ git remote add origin [email protected]:BAD_URL
$ git remote show origin
ERROR:gitosis.serve.main:Repository read access denied
fatal: The remote end hung up unexpectedly
$ 

You can then spawn a 'git remote update' on a thread to fetch all branches. (essentially doing a clone).

[joelstransky]

Where is gitifier cloning to? I'm uncomfortable about two clones of the same repo on my system.

[mackuba]

It's mentioned in the readme, the clones are in ~/Library/Caches/net.psionides.Gitifier. Why is that a problem, is your repository very big?

[joelstransky]

Ah ok. Didn't realize gitifier was only cloning the .git folder. Not all of the files. My concern was duplicate files showing up in a spotlight search or similar task. Does it pull every time it checks? We've had problems in the past with repo's becoming huge due to binary representations of entire graphic libraries existing in every commit object.

[mackuba]

Specifically, it does a git fetch. This updates the remote/* branch references, and if they did change, then git log is called in the range returned by git fetch.

I'm not that familiar with the internal structure of git repos, but I imagine that if those binaries don't change, there would be only one copy stored inside .git...