File name too long when trying to get Terraform module from Gitlab

[dtrouillet]

Describe the issue When I try to run checkov with download-external-modules to true : checkov -d . -o junitxml --external-checks-git https://gitlab.mycompany.com/mygroup/policy-as-code.git --download-external-modules True | tee checkov.test.xml the module can't be downloaded du to "Too Long File Name".

I have this issue because the CI Job Token is in the file name : failed to get dpi/terraform-rancher-project/local because of [Errno 36] File name too long: '/builds/mygroup/myapp/.external_modules/gitlab.mycompany.com/mygroup/terraform-mymodule-project/local/1.0.0/module_source.eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbiI6IjY0X1NSa3BDV3Jjd0NYOGFBcnM4dl94IiwianRpIjoiODA2MTg5ZDgtMjdmOC00ZTFlLThkNTYtMTAzZWQ2YzQzNGI2IiwiaWF0IjoxNjk0MTkxNjAwLCJuYmYiOjE2OTQxOTE1OTUsImV4cCI6MTY5NDE5MTkwMH0.w12LAR7HAgbPfW2nAr-RhrWsnElvCpdqzIXYwPtizlQ&archive'.

Is it possible to shrink the file name? I think the trouble can be fix in this file but I'm not sure : https://github.com/bridgecrewio/checkov/blob/b978759fbed21d0e23a9698e8bc8840757f244d1/checkov/common/goget/registry/get_registry.py#L20C12-L20C12

Regards

[fredgate]

I have the same problem.

Why does checkov need to download the external modules whereas they have already been retrieved by a terraform init and they are present in the .terraform/modules folder of the project ?

[gruebel]

@fredgate The reason why we download the modules on our own, because not everyone can run terraform init you wouldn't be able to scan subfolders, etc.

That being said, we will add experimental support of leveraging the Terraform downloaded modules soonish #5635 but without nested modules support at the beginning. This will become the default behaviour after we did more internal testing.

@dtrouillet regarding your actual question, in theory it should be possible, but it needs to be tested properly.

[patrcoff]

Is this still desired? @dtrouillet

The fix would likely be in the specific loader file for gitlab - I'm currently looking at an issue I have with the generic git loader (currently looking through the issues and PRs to see it's not already logged but I have an idea of how to fix my issue, will be looking more closely at the weekend) - I might be able to also have a look at this one as it's related to mine, if it's still desired?

[dtrouillet]

Hi @patrcoff ,

Yes it still desired. Can you share your issue please?

Thanks!

[patrcoff]

@dtrouillet - sorry for the slow response, had a few busy days with the family (pregnant wife, young child, lots of pets).

I've created my issue here

I believe the logic to my fix (works for me but yet to be properly tested) would also fix your issue but I didn't get a chance this weekend to properly go deep into it and verify my logic, run tests etc. I'll try and get it done this week though outside of day job hours. Then at least you'll have a fork to work with, and fingers crossed it'll get succesfully PR'd in time.

[dtrouillet]

@patrcoff , No problem!! Thanks for your work!