gitlabber icon indicating copy to clipboard operation
gitlabber copied to clipboard

Gitlabber failing with 404 Group Not Found

Open MaxRink opened this issue 2 years ago • 4 comments

Describe the bug Im trying to recursively clone all gitlab projects accessible to me but gitlabber fails in the fetching project stage with 404 Group not found. It looks like the GitLab API returns that i have access to groups that i actually don't have access to To Reproduce

gitlabber -t token -u https://gitlab.aa.aaa.de -a exclude -r  -m ssh -n path  -x "/schiff/archived**,/schiff/demo**,/schiff/platform/tenants/schiff-keystore,/schiff/platform/infrastructure/dotfiles,/rms2/**,**rms**" . --verbose
2021-12-15 12:11:43,120 - gitlabber.cli - DEBUG - verbose=[True], print=[False], log level set to [10] level
2021-12-15 12:11:43,141 - gitlabber.cli - DEBUG - Reading projects tree from gitlab at [https://gitlab.aa.aaa.de]
2021-12-15 12:11:43,141 - gitlabber.gitlab_tree - DEBUG - Loading projects tree gitlab server [https://gitlab.aa.aaa.de]
2021-12-15 12:11:43,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gitlab.aa.aaa.de:443
2021-12-15 12:11:43,339 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:43,453 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23850/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:43,635 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23868 HTTP/1.1" 200 None
2021-12-15 12:11:43,746 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23868/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:43,917 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23868/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:43,985 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23850/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:44,103 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3146/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:44,423 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3451 HTTP/1.1" 200 None
2021-12-15 12:11:44,549 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3451/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:44,842 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3744 HTTP/1.1" 200 None
2021-12-15 12:11:44,937 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3744/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:45,158 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3744/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:45,303 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3469 HTTP/1.1" 200 None
2021-12-15 12:11:45,438 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3469/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:45,518 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3469/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:45,685 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3456 HTTP/1.1" 200 None
2021-12-15 12:11:45,813 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3456/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:46,106 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3457 HTTP/1.1" 200 None
2021-12-15 12:11:46,219 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3457/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:46,403 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3457/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:46,632 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3456/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:46,827 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3468 HTTP/1.1" 200 None
2021-12-15 12:11:46,934 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3468/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,017 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3468/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,165 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3465 HTTP/1.1" 200 None
2021-12-15 12:11:47,265 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3465/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,344 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3465/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,491 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3466 HTTP/1.1" 200 None
2021-12-15 12:11:47,589 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3466/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,671 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3466/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:47,823 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5385 HTTP/1.1" 200 None
2021-12-15 12:11:47,924 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5385/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:48,056 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5385/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:48,230 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3470 HTTP/1.1" 200 None
2021-12-15 12:11:48,333 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3470/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:48,424 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3470/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:48,568 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5638 HTTP/1.1" 200 None
2021-12-15 12:11:48,672 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5638/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:48,748 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/5638/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:49,013 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3451/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:49,285 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3452 HTTP/1.1" 200 None
2021-12-15 12:11:49,432 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3452/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:49,634 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3452/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:49,864 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11818 HTTP/1.1" 200 None
2021-12-15 12:11:49,966 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11818/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:50,109 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/36199 HTTP/1.1" 200 None
2021-12-15 12:11:50,213 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/36199/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:50,333 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/36199/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:50,539 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11818/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:50,650 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3190 HTTP/1.1" 200 None
2021-12-15 12:11:50,762 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3190/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:51,368 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3189 HTTP/1.1" 200 None
2021-12-15 12:11:51,468 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3189/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:51,938 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3189/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:52,629 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3187 HTTP/1.1" 200 None
2021-12-15 12:11:52,747 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3187/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:52,899 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15903 HTTP/1.1" 200 None
2021-12-15 12:11:52,992 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15903/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:53,121 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15903/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:53,500 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3187/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:53,898 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3187/projects?archived=false&id=3187&include_subgroups=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false&with_security_reports=false&with_shared=true&archived=False HTTP/1.1" 200 None
2021-12-15 12:11:53,973 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3190/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:54,143 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16797 HTTP/1.1" 200 None
2021-12-15 12:11:54,237 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16797/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:54,545 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16799 HTTP/1.1" 200 None
2021-12-15 12:11:54,644 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16799/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:54,880 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16799/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:55,085 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16798 HTTP/1.1" 200 None
2021-12-15 12:11:55,183 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16798/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:55,649 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16798/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:55,776 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/16797/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:56,006 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3146/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:56,094 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/242/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:56,413 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15236 HTTP/1.1" 200 None
2021-12-15 12:11:56,503 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15236/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:56,741 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/15236/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:56,943 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/242/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:57,044 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/6915/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:57,234 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/6915/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:57,317 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3197/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:57,497 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/3197/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:57,601 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2939/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:57,896 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/9694 HTTP/1.1" 200 None
2021-12-15 12:11:58,011 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/9694/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:58,262 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/9694/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:58,558 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2939/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:58,701 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups?archived=False&order_by=name&owned=false&page=2&per_page=20&sort=asc&statistics=false&with_custom_attributes=false&archived=False HTTP/1.1" 200 None
2021-12-15 12:11:58,814 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18573/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:58,919 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18581 HTTP/1.1" 200 None
2021-12-15 12:11:59,034 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18581/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:59,216 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18583 HTTP/1.1" 200 None
2021-12-15 12:11:59,335 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18583/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:59,459 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18583/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:59,542 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18581/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:59,625 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/18573/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:11:59,734 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2707/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:11:59,912 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2710 HTTP/1.1" 200 None
2021-12-15 12:12:00,023 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2710/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:00,178 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2710/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:00,256 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/2707/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:00,360 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/6314/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:00,610 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11253 HTTP/1.1" 200 None
2021-12-15 12:12:00,722 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11253/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:00,878 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37150 HTTP/1.1" 200 None
2021-12-15 12:12:00,984 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37150/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:01,167 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37165 HTTP/1.1" 200 None
2021-12-15 12:12:01,264 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37165/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:01,428 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37165/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:01,626 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37166 HTTP/1.1" 200 None
2021-12-15 12:12:01,723 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37166/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:01,879 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37166/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:02,013 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/37150/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:02,264 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11501 HTTP/1.1" 200 None
2021-12-15 12:12:02,359 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11501/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:02,628 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11501/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:02,856 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23018 HTTP/1.1" 200 None
2021-12-15 12:12:02,952 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23018/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:03,132 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/23018/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:03,345 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/11253/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:03,437 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/6314/projects?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:03,538 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/194/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:03,872 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/33926 HTTP/1.1" 200 None
2021-12-15 12:12:03,959 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/33926/subgroups?archived=False HTTP/1.1" 200 2
2021-12-15 12:12:04,210 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/33926/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:04,353 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/194/projects?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:04,485 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/22817/subgroups?archived=False HTTP/1.1" 200 None
2021-12-15 12:12:04,550 - urllib3.connectionpool - DEBUG - https://gitlab.aa.aaa.de:443 "GET /api/v4/groups/34573 HTTP/1.1" 404 33
Traceback (most recent call last):
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlab/exceptions.py", line 287, in wrapped_f
    return f(*args, **kwargs)
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlab/mixins.py", line 112, in get
    server_data = self.gitlab.http_get(path, **kwargs)
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlab/client.py", line 618, in http_get
    result = self.http_request(
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlab/client.py", line 584, in http_request
    raise gitlab.exceptions.GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 404: 404 Group Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/maxr/.local/bin/gitlabber", line 8, in <module>
    sys.exit(main())
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlabber/cli.py", line 43, in main
    tree.load_tree()
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlabber/gitlab_tree.py", line 144, in load_tree
    self.load_gitlab_tree()
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlabber/gitlab_tree.py", line 127, in load_gitlab_tree
    self.get_subgroups(group, node)
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlabber/gitlab_tree.py", line 112, in get_subgroups
    subgroup = self.gitlab.groups.get(subgroup_def.id)
  File "/home/maxr/.local/lib/python3.9/site-packages/gitlab/exceptions.py", line 289, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabGetError: 404: 404 Group Not Found

Expected behavior Gitlabber is able to build the tree correctly Versions (please complete the following information):

  • OS: [e.g. iOS] and version Manjaro
  • Python Version [e.g. 2.6]: 3.9
  • Gitlab type [e.g. host, cloud] and Version [e.g. 14.0] : GitLab Enterprise Edition 14.5.2-ee
  • Gitlabber version [e.g. 1.18] 1.1.8

Additional context Add any other context about the problem here.

MaxRink avatar Dec 15 '21 11:12 MaxRink

You can check if Pull Request works for you. Pls leave a comment if it works.

devstek avatar Dec 23 '21 07:12 devstek

looks like it does @devstek

MaxRink avatar Jan 28 '22 12:01 MaxRink

Or actually it doesnt I apparently used the wrong api key to reporduce it

2022-01-28 13:33:56,721 - urllib3.connectionpool - DEBUG - https://gitlab.devops.telekom.de:443 "GET /api/v4/groups/34573 HTTP/1.1" 404 33
Traceback (most recent call last):
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/exceptions.py", line 311, in wrapped_f
    return f(*args, **kwargs)
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/mixins.py", line 111, in get
    server_data = self.gitlab.http_get(path, **kwargs)
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/client.py", line 761, in http_get
    result = self.http_request(
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/client.py", line 727, in http_request
    raise gitlab.exceptions.GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 404: 404 Group Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/gitlab_tree.py", line 122, in get_subgroups
    subgroup = self.gitlab.groups.get(subgroup_def.id)
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/v4/objects/groups.py", line 323, in get
    return cast(Group, super().get(id=id, lazy=lazy, **kwargs))
  File "/home/maxr/.local/lib/python3.10/site-packages/gitlab/exceptions.py", line 313, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabGetError: 404: 404 Group Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/__main__.py", line 2, in <module>
    main()
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/cli.py", line 43, in main
    tree.load_tree()
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/gitlab_tree.py", line 161, in load_tree
    self.load_gitlab_tree()
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/gitlab_tree.py", line 144, in load_gitlab_tree
    self.get_subgroups(group, node)
  File "/home/maxr/git/GitHub/ezbz/gitlabber/gitlabber2/gitlab_tree.py", line 125, in get_subgroups
    log.error(f"Error retrieving group {subgroup.name} (ID: {subgroup.id}): {error}. Check your permissions as you may not have access to it.")
UnboundLocalError: local variable 'subgroup' referenced before assignment

MaxRink avatar Jan 28 '22 12:01 MaxRink

I also get "404 Group Not Found" from gitlabber 1.1.9 against GitLab 15.4.2-ee.

It first calls https://gitlab/api/v4/groups without a min_access_level. The returned JSON gives no indication which groups the authenticated user has access to. It then tries to list the subgroups of every group. GitLab uses 404 even if it means 403, probably to avoid leaking information. So boom.

tewe avatar Oct 19 '22 08:10 tewe