libgit2sharp icon indicating copy to clipboard operation
libgit2sharp copied to clipboard

RetrieveStatus take so long time about 5s

Open sumayn opened this issue 7 years ago • 13 comments

I‘m used RetrieveStatus method. it take so long time . _repo.RetrieveStatus(new StatusOptions()) libgit2sharp version 0.24.0

How should I deal with this problem?

Thank u

sumayn avatar Apr 20 '18 08:04 sumayn

Does git status take a long time? What about git status --ignored?

ethomson avatar Apr 20 '18 08:04 ethomson

so fast do git status or git status --ignored

but use libgit2sharp api _repo.RetrieveStatus(new StatusOptions()) so slowly

sumayn avatar Apr 20 '18 09:04 sumayn

Let me describe the same issue for me. When libgit2 tries to get status of files (modified or not), library compares mtime/ctime flags from index to folder's attributes. If folder is modified, then libgit2 searches the contents of each file/folder in modified folder to figure out if file was modified or not. I don't know why, but on some machines folder attributes always diff from information written in git index. That forces library to watch every file in repository. So, many files in repository slows down performance of RetrieveStatus.

One of our company computer calls RetrieveStatus for 40 seconds. On another computers that tooks only 3-4 seconds.

Snegovikufa avatar Jun 05 '18 08:06 Snegovikufa

What version of git? Are you sure the mtime and ctime in the index match the filesystem? In particular the high resolution bits?

ethomson avatar Jun 05 '18 08:06 ethomson

That issue was not on my own computer. Git version is something like 2.10. We started to monitor file system with Process Monitor (https://technet.microsoft.com/ru-ru/sysinternals/processmonitor.aspx).

On "bad" computer there only records like at screenshot. All files in repository was opened. procmon64_2018-06-05_13-38-42

I compared the output of git ls-files --debug between two computers and saw that there were differences in mtime flag. I tried to understand what mtime means. mtime is the "modified time", right? I opened libgit2 source code and found that if comment mtime check, that fixes slow performance on "bad" computer:


----------------------- src/diff_generate.c -----------------------------
index e11cbe4e4..b3e9a37c0 100644
@@ -817,8 +817,9 @@ static int maybe_modified(
 			modified_uncertain =
 				(oitem->file_size <= 0 && nitem->file_size > 0);
 		}
-		else if (!git_index_time_eq(&oitem->mtime, &nitem->mtime) ||
-			(use_ctime && !git_index_time_eq(&oitem->ctime, &nitem->ctime)) ||
+		else if (
+			/*!git_index_time_eq(&oitem->mtime, &nitem->mtime) ||
+			(use_ctime && !git_index_time_eq(&oitem->ctime, &nitem->ctime)) ||*/
 			oitem->ino != nitem->ino ||
 			oitem->uid != nitem->uid ||
 			oitem->gid != nitem->gid ||
@@ -859,7 +860,7 @@ static int maybe_modified(
 	 * and an add of the new so that consumers can act accordingly (eg,
 	 * checkout will update the case on disk.)
 	 */
-	if (DIFF_FLAG_IS_SET(diff, GIT_DIFF_IGNORE_CASE) &&
+	/*if (DIFF_FLAG_IS_SET(diff, GIT_DIFF_IGNORE_CASE) &&
 		DIFF_FLAG_IS_SET(diff, GIT_DIFF_INCLUDE_CASECHANGE) &&
 		strcmp(oitem->path, nitem->path) != 0) {
 
@@ -867,7 +868,7 @@ static int maybe_modified(
 			error = diff_delta__from_one(diff, GIT_DELTA_ADDED, NULL, nitem);
 
 		return error;
-	}
+	}*/
 
 	return diff_delta__from_two(
 		diff, status, oitem, omode, nitem, nmode,


Snegovikufa avatar Jun 05 '18 08:06 Snegovikufa

Are you copying this repository between two computers directly? Or sharing it on a network drive or something?

ethomson avatar Jun 05 '18 09:06 ethomson

We clone repository from remote TFS repository.

Snegovikufa avatar Jun 05 '18 09:06 Snegovikufa

Well, here's what I'm trying to understand: if the mtime is different from the cache, yes, libgit2 (and git) will inspect the file to determine if it's changed. So after cloning, something is touching that filestamp on disk so that it doesn't match the cache. Any idea what's doing that?

ethomson avatar Jun 05 '18 09:06 ethomson

something is touching that filestamp on disk so that it doesn't match the cache.

I had that idea too but I don't know what modifies timestamps on disk.

Is there any tool to synchronize git cache and timestamp on disk?

Snegovikufa avatar Jun 05 '18 09:06 Snegovikufa

Yes. git status will do that once it's determined that you haven't made any changes.

ethomson avatar Jun 05 '18 10:06 ethomson

git status --debug on my machine:

...
env/DefaultTemplates/en-US/MS Excel template/MS Excel template.json
  ctime: 1510654479:766007800
  mtime: 1510654479:766007800
  dev: 0	ino: 0
  uid: 0	gid: 0
  size: 138	flags: 0
...

On "bad" machine git version is 2.8.1. I confused with :0 value in ctime and mtime. Also, I don't know what 'dev' means

env/DefaultTemplates/en-US/MS Excel template/MS Excel template.json
  ctime: 1522231113:0
  mtime: 1522231113:0
  dev: 2	ino: 0
  uid: 0	gid: 0
  size: 138	flags: 0

Snegovikufa avatar Jun 06 '18 12:06 Snegovikufa

This is the same issue for me. Takes a very long time while git status is instant.

kboom avatar Jan 27 '23 09:01 kboom

Was there any solution for those who were affected in the past?

What is the C# code with all option flags to mimic git status?

PulsarFX avatar Jul 12 '24 10:07 PulsarFX