gitpython-developers
RemoteProgress update broken while fetching
Used version
Git: 2.34.1
GitPython: 3.1.43
Python: 3.10.12
FYI, progress bar rendered with rich (13.7.1)
Issue
RemoteProgress update method is called with ill-formed parameters when use with remote.fetch and thus leading to broken progress bar. This is working well with clone and/or clone_from. Update is called only once per git operation (e.g. COUNTING, COMPRESSING) with BEGIN and END opcode flag set and all messages concatenated.
It seems that remote progress lines are not parsed correctly while fetching remote.
Correct behavior with clone
Cloning git repository mypackage (revision=branch1)...
Counting ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% eta 0:00:00
Compressing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% eta 0:00:00
Receiving ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% eta 0:00:00 6.45 MiB | 12.85 MiB/s
Resolving ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% eta 0:00:00
Broken progress bar with fetch
Updating mypackage (revision=main)...
Count… remote: Counting objects: 1% (6/592) remote: Counting objects: 2% (12/592) remote: Counting objects: 3% (18/592) remote: Counting objects: 4% (24/59…
Compr… remote: Compressing objects: 1% (3/240) remote: Compressing objects: 2% (5/240) remote: Compressing objects: 3% (8/240) remote: Compressing objects: …
Recei… Receiving objects: 77% (456/592)Receiving objects: 78% (462/592)Receiving objects: 79% (468/592)Receiving objects: 80% (474/592)Receiving objects: 81% (480/592)Receiving ob…
Resol… Resolving deltas: 1% (5/406)Resolving deltas: 2% (9/406)Resolving deltas: 3% (13/406)Resolving deltas: 4% (17/406)Resolving deltas: 5% (21/406)Resolving deltas: 6% (…
Thanks for reporting.
Parsing should happen in real-time even though this looks like it's not able to separate the lines, or it sees all lines at once for some reason. Maybe it's related to synchronization also in the relation to how the program that uses GitPython is written, even though it seems to work with clone and clone_from.
So maybe it would be possible to find a difference between the fetch and clone, which to my knowledge use the same underlying engine.
With GitPython 3.1.40, it works well, progress bar is not broken on fetch sub command. I'll try the two next patch level release.
Here dumps of handle process output while fetching:
b'POST git-upload-pack (294 bytes)\n'
b'POST git-upload-pack (958 bytes)\n'
b'remote: Enumerating objects: 592, done. \n'
b'remote: Counting objects: 0% (1/592) \rremote: Counting objects: 1% (6/592) \rremote: Counting objects: 2% (12/592) \rremote: Counting objects: 3% (18/592) \rremote: Counting objects: 4% (24/592) \rremote: Counting objects: 5% (30/592) \rremote: Counting objects: 6% (36/592) \rremote: Counting objects: 7% (42/592) \rremote: Counting objects: 8% (48/592) \rremote: Counting objects: 9% (54/592) \rremote: Counting objects: 10% (60/592) \rremote: Counting objects: 11% (66/592) \rremote: Counting objects: 12% (72/592) \rremote: Counting objects: 13% (77/592) \rremote: Counting objects: 14% (83/592) \rremote: Counting objects: 15% (89/592) \rremote: Counting objects: 16% (95/592) \rremote: Counting objects: 17% (101/592) \rremote: Counting objects: 18% (107/592) \rremote: Counting objects: 19% (113/592) \rremote: Counting objects: 20% (119/592) \rremote: Counting objects: 21% (125/592) \rremote: Counting objects: 22% (131/592) \rremote: Counting objects: 23% (137/592) \rremote: Counting objects: 24% (143/592) \rremote: Counting objects: 25% (148/592) \rremote: Counting objects: 26% (154/592) \rremote: Counting objects: 27% (160/592) \rremote: Counting objects: 28% (166/592) \rremote: Counting objects: 29% (172/592) \rremote: Counting objects: 30% (178/592) \rremote: Counting objects: 31% (184/592) \rremote: Counting objects: 32% (190/592) \rremote: Counting objects: 33% (196/592) \rremote: Counting objects: 34% (202/592) \rremote: Counting objects: 35% (208/592) \rremote: Counting objects: 36% (214/592) \rremote: Counting objects: 37% (220/592) \rremote: Counting objects: 38% (225/592) \rremote: Counting objects: 39% (231/592) \rremote: Counting objects: 40% (237/592) \rremote: Counting objects: 41% (243/592) \rremote: Counting objects: 42% (249/592) \rremote: Counting objects: 43% (255/592) \rremote: Counting objects: 44% (261/592) \rremote: Counting objects: 45% (267/592) \rremote: Counting objects: 46% (273/592) \rremote: Counting objects: 47% (279/592) \rremote: Counting objects: 48% (285/592) \rremote: Counting objects: 49% (291/592) \rremote: Counting objects: 50% (296/592) \rremote: Counting objects: 51% (302/592) \rremote: Counting objects: 52% (308/592) \rremote: Counting objects: 53% (314/592) \rremote: Counting objects: 54% (320/592) \rremote: Counting objects: 55% (326/592) \rremote: Counting objects: 56% (332/592) \rremote: Counting objects: 57% (338/592) \rremote: Counting objects: 58% (344/592) \rremote: Counting objects: 59% (350/592) \rremote: Counting objects: 60% (356/592) \rremote: Counting objects: 61% (362/592) \rremote: Counting objects: 62% (368/592) \rremote: Counting objects: 63% (373/592) \rremote: Counting objects: 64% (379/592) \rremote: Counting objects: 65% (385/592) \rremote: Counting objects: 66% (391/592) \rremote: Counting objects: 67% (397/592) \rremote: Counting objects: 68% (403/592) \rremote: Counting objects: 69% (409/592) \rremote: Counting objects: 70% (415/592) \rremote: Counting objects: 71% (421/592) \rremote: Counting objects: 72% (427/592) \rremote: Counting objects: 73% (433/592) \rremote: Counting objects: 74% (439/592) \rremote: Counting objects: 75% (444/592) \rremote: Counting objects: 76% (450/592) \rremote: Counting objects: 77% (456/592) \rremote: Counting objects: 78% (462/592) \rremote: Counting objects: 79% (468/592) \rremote: Counting objects: 80% (474/592) \rremote: Counting objects: 81% (480/592) \rremote: Counting objects: 82% (486/592) \rremote: Counting objects: 83% (492/592) \rremote: Counting objects: 84% (498/592) \rremote: Counting objects: 85% (504/592) \rremote: Counting objects: 86% (510/592) \rremote: Counting objects: 87% (516/592) \rremote: Counting objects: 88% (521/592) \rremote: Counting objects: 89% (527/592) \rremote: Counting objects: 90% (533/592) \rremote: Counting objects: 91% (539/592) \rremote: Counting objects: 92% (545/592) \rremote: Counting objects: 93% (551/592) \rremote: Counting objects: 94% (557/592) \rremote: Counting objects: 95% (563/592) \rremote: Counting objects: 96% (569/592) \rremote: Counting objects: 97% (575/592) \rremote: Counting objects: 98% (581/592) \rremote: Counting objects: 99% (587/592) \rremote: Counting objects: 100% (592/592) \rremote: Counting objects: 100% (592/592), done. \n'
Output with working progress bar (v3.1.40):
remote: Enumerating objects: 150, done.
remote: Counting objects: 0% (1/150)
remote: Counting objects: 1% (2/150)
remote: Counting objects: 2% (3/150)
remote: Counting objects: 3% (5/150)
remote: Counting objects: 4% (6/150)
which to my knowledge use the same underlying engine
underlying engine is the same, but input stream parsed is not.
Okay, seems that i found something interresting. I'll push a pull request (w/ working progress bar)