Not scanning all files k8s-manifest in directory

[Poweranimal]

Describe the issue Checkov doesn't scan all 8s-manifest files in a directory, if there are too many files/file-content present.

Examples I have the following directory structure:

ls -d ./dist/*

./dist/0000-prometheuscustomresourcedefinitions-chart-c8e9cde4.k8s.yaml
./dist/0001-opentelemetrycustomresourcedefinitions-chart-c8a9689e.k8s.yaml
./dist/0002-consulcustomresourcedefinitions-chart-c8610464.k8s.yaml
./dist/0003-consulapigatewaycustomresourcedefinitions-chart-c804f07a.k8s.yaml
./dist/0004-gatewayapicustomresourcedefinitions-chart-c8c19908.k8s.yaml
./dist/0005-awsloadbalancercontrollercustomresourcedefinitio-chart-c8205ad9.k8s.yaml
./dist/0006-metricsserver-chart-c82a0c31.k8s.yaml
./dist/0007-certmanager-chart-c848915d.k8s.yaml
./dist/0008-gatewayapi-chart-c8321308.k8s.yaml
./dist/0009-configsyncer-chart-c8de2343.k8s.yaml
./dist/0010-opentelemetry-chart-c8d5ed06.k8s.yaml
./dist/0011-vault-chart-c8dea291.k8s.yaml
./dist/0012-consul-chart-c8e8de17.k8s.yaml
./dist/0013-consul-postsetupchart-c8edc1ca.k8s.yaml
./dist/0014-minio-chart-c8a61440.k8s.yaml
./dist/0015-redis-chart-c8d905d9.k8s.yaml
./dist/0016-tempo-chart-c85c08f2.k8s.yaml
./dist/0017-loki-chart-c893c723.k8s.yaml
./dist/0018-promtail-chart-c8ac1752.k8s.yaml
./dist/0019-mimir-chart-c8947681.k8s.yaml
./dist/0020-prometheus-chart-c81fe05e.k8s.yaml
./dist/0021-grafana-chart-c8e72439.k8s.yaml
./dist/0022-zookeeper-chart-c814ec51.k8s.yaml
./dist/0023-kafka-component-chart-c8dafe4b.k8s.yaml
./dist/0024-postgres-component-chart-c8ebcea4.k8s.yaml

Checkov exists scanning files after the file ./dist/0011-vault-chart-c8dea291.k8s.yaml with error code 0.

Version (please complete the following information):

• Checkov Version 2.1.204

Additional context Add any other context about the problem here.

[gruebel]

hey @Poweranimal thanks for reaching out. Can you rerun the checkov command you used, but adjusting the log level and share what you get there.

LOG_LEVEL=DEBUG checkov -d xyz

[Poweranimal]

@gruebel Thank you very much for your quick response.

Running: LOG_LEVEL=DEBUG checkov -d ./dist produces the following debug logs. I have only attached the stderr, because including the stdout would have increased the log size a lot. From the stderr I excluded the middle part describing the checks it runs in order to reduce the log size.

Logs

2022-09-12 23:21:51,384 [MainThread  ] [DEBUG]  Leveraging the bundled IAM Definition.
2022-09-12 23:21:51,384 [MainThread  ] [DEBUG]  Leveraging the IAM definition at /var/home/poweranimal/.local/lib/python3.10/site-packages/policy_sentry/shared/data/iam-definition.json
2022-09-12 23:21:51,564 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/bicep/checks/graph_checks
2022-09-12 23:21:51,640 [MainThread  ] [DEBUG]  Popen(['git', 'version'], cwd=/var/home/poweranimal/Documents/GitHub/gl-solution, universal_newlines=False, shell=None, istream=None)
2022-09-12 23:21:51,643 [MainThread  ] [DEBUG]  Popen(['git', 'version'], cwd=/var/home/poweranimal/Documents/GitHub/gl-solution, universal_newlines=False, shell=None, istream=None)
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  No API key present; setting include_all_checkov_policies to True
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Checkov version: 2.1.204
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Python executable: /usr/bin/python3
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Python version: 3.10.6 (main, Aug  2 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Checkov executable (argv[0]): /var/home/poweranimal/.local/bin/checkov
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Command Line Args:   -d ./dist
Config File (/var/home/poweranimal/Documents/GitHub/gl-solution/.checkov.yaml):
  compact:           True
  framework:         ['kubernetes']
  skip-check:        ['CKV_K8S_43', 'CKV_K8S_15']
Defaults:
  --branch:          master
  --download-external-modules:False
  --external-modules-download-path:.external_modules
  --evaluate-variables:True
  --secrets-scan-file-type:[]
  --black-list-secret-scan:[]

2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  Resultant set of frameworks (removing skipped frameworks): kubernetes
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  kubernetes_runner declares no system dependency checks required.
2022-09-12 23:21:51,848 [MainThread  ] [DEBUG]  No API key found. Scanning locally only.
2022-09-12 23:21:52,431 [MainThread  ] [DEBUG]  Got checkov mappings and guidelines from Bridgecrew platform
2022-09-12 23:21:52,432 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/terraform/checks/graph_checks
2022-09-12 23:21:52,432 [MainThread  ] [DEBUG]  Searching through ['aws', 'azure', 'gcp', '__pycache__'] and ['__init__.py']
2022-09-12 23:21:52,432 [MainThread  ] [DEBUG]  Searching through [] and ['ALBProtectedByWAF.yaml', 'ALBRedirectsHTTPToHTTPS.yaml', 'AMRClustersNotOpenToInternet.yaml', 'APIGWLoggingLevelsDefinedProperly.yaml', 'APIProtectedByWAF.yaml', 'AWSNATGatewaysshouldbeutilized.yaml', 'AWSSSMParameterShouldBeEncrypted.yaml', 'AppLoadBalancerTLS12.yaml', 'AppSyncProtectedByWAF.yaml', 'AutoScalingEnableOnDynamoDBTables.yaml', 'AutoScallingEnabledELB.yaml', 'CloudFrontHasResponseHeadersPolicy.yaml', 'CloudtrailHasCloudwatch.yaml', 'CodecommitApprovalRulesAttached.yaml', 'EBSAddedBackup.yaml', 'EFSAddedBackup.yaml', 'EIPAllocatedToVPCAttachedEC2.yaml', 'EncryptedEBSVolumeOnlyConnectedToEC2s.yaml', 'GuardDutyIsEnabled.yaml', 'HTTPNotSendingPasswords.yaml', 'IAMGroupHasAtLeastOneUser.yaml', 'IAMUserHasNoConsoleAccess.yaml', 'IAMUsersAreMembersAtLeastOneGroup.yaml', 'PostgresDBHasQueryLoggingEnabled.yaml', 'PostgresRDSHasQueryLoggingEnabled.yaml', 'RDSClusterHasBackupPlan.yaml', 'Route53ARecordAttachedResource.yaml', 'Route53ZoneEnableDNSSECSigning.yaml', 'Route53ZoneHasMatchingQueryLog.yaml', 'S3BucketEncryption.yaml', 'S3BucketHasPublicAccessBlock.yaml', 'S3BucketLogging.yaml', 'S3BucketReplicationConfiguration.yaml', 'S3BucketVersioning.yaml', 'S3KMSEncryptedByDefault.yaml', 'S3PublicACLRead.yaml', 'S3PublicACLWrite.yaml', 'SGAttachedToResource.yaml', 'SubnetHasACL.yaml', 'VPCHasFlowLog.yaml', 'VPCHasRestrictedSG.yaml', 'WAF2HasLogs.yaml']
2022-09-12 23:21:52,588 [MainThread  ] [DEBUG]  Searching through [] and ['AccessToPostgreSQLFromAzureServicesIsDisabled.yaml', 'ApplicationGatewayEnablesWAF.yaml', 'AzureActiveDirectoryAdminIsConfigured.yaml', 'AzureAntimalwareIsConfiguredWithAutoUpdatesForVMs.yaml', 'AzureDataFactoriesEncryptedWithCustomerManagedKey.yaml', 'AzureMSSQLServerHasSecurityAlertPolicy.yaml', 'AzureNetworkInterfacePublicIPAddressId.yaml', 'AzureStorageAccountsUseCustomerManagedKeyForEncryption.yaml', 'AzureSynapseWorkspacesHaveNoIPFirewallRulesAttached.yaml', 'AzureUnattachedDisksAreEncrypted.yaml', 'CognitiveServicesCustomerManagedKey.yaml', 'DataExplorerEncryptionUsesCustomKey.yaml', 'MSQLenablesCustomerManagedKey.yaml', 'PGSQLenablesCustomerManagedKey.yaml', 'SQLServerAuditingEnabled.yaml', 'SQLServerAuditingRetention90Days.yaml', 'StorageContainerActivityLogsNotPublic.yaml', 'StorageCriticalDataEncryptedCMK.yaml', 'StorageLoggingIsEnabledForBlobService.yaml', 'StorageLoggingIsEnabledForTableService.yaml', 'VAconfiguredToSendReports.yaml', 'VAconfiguredToSendReportsToAdmins.yaml', 'VAisEnabledInStorageAccount.yaml', 'VAsetPeriodicScansOnSQL.yaml', 'VMHasBackUpMachine.yaml', 'VirtualMachinesUtilizingManagedDisks.yaml']
2022-09-12 23:21:52,642 [MainThread  ] [DEBUG]  Searching through [] and ['DisableAccessToSqlDBInstanceForRootUsersWithoutPassword.yaml', 'GCPAuditLogsConfiguredForAllServicesAndUsers.yaml', 'GCPContainerRegistryReposAreNotPubliclyAccessible.yaml', 'GCPKMSCryptoKeysAreNotPubliclyAccessible.yaml', 'GCPKMSKeyRingsAreNotPubliclyAccessible.yaml', 'GCPLogBucketsConfiguredUsingLock.yaml', 'GCPProjectHasNoLegacyNetworks.yaml', 'GKEClustersAreNotUsingDefaultServiceAccount.yaml', 'ServiceAccountHasGCPmanagedKey.yaml']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Searching through [] and ['__init__.cpython-310.pyc']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/cloudformation/checks/graph_checks
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Searching through ['__pycache__'] and ['__init__.py']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Searching through [] and ['__init__.cpython-310.pyc']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/kubernetes/checks/graph_checks
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Searching through ['__pycache__'] and ['__init__.py']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Searching through [] and ['__init__.cpython-310.pyc']
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/bicep/checks/graph_checks
2022-09-12 23:21:52,671 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/terraform_plan/checks/graph_checks
2022-09-12 23:21:52,674 [MainThread  ] [DEBUG]  Filtered list of policies: []
2022-09-12 23:21:53,080 [MainThread  ] [DEBUG]  Cannot read file contents: ./dist/0000-prometheuscustomresourcedefinitions-chart-c8e9cde4.k8s.yaml - is it a yaml?
2022-09-12 23:21:53,431 [MainThread  ] [INFO ]  creating Kubernetes graph
2022-09-12 23:21:53,541 [MainThread  ] [INFO ]  Successfully created Kubernetes graph

<REMOVED LOGS TO REDUCE SIZE>

2022-09-12 23:21:54,145 [MainThread  ] [DEBUG]  Loading external checks from /var/home/poweranimal/.local/lib/python3.10/site-packages/checkov/kubernetes/checks/graph_checks
2022-09-12 23:21:54,145 [MainThread  ] [DEBUG]  Searching through ['__pycache__'] and ['__init__.py']
2022-09-12 23:21:54,145 [MainThread  ] [DEBUG]  Searching through [] and ['__init__.cpython-310.pyc']
2022-09-12 23:21:54,147 [MainThread  ] [DEBUG]  Getting exit code for report kubernetes
2022-09-12 23:21:54,147 [MainThread  ] [DEBUG]  Soft fail severity threshold: None
2022-09-12 23:21:54,147 [MainThread  ] [DEBUG]  Soft fail checks: []
2022-09-12 23:21:54,148 [MainThread  ] [DEBUG]  Hard fail severity threshold: None
2022-09-12 23:21:54,148 [MainThread  ] [DEBUG]  Hard fail checks: []
2022-09-12 23:21:54,148 [MainThread  ] [DEBUG]  Use enforcement rules is FALSE
2022-09-12 23:21:54,148 [MainThread  ] [DEBUG]  In get_exit_code; exit code thresholds: {'soft_fail': False, 'soft_fail_checks': [], 'soft_fail_threshold': None, 'hard_fail_checks': [], 'hard_fail_threshold': None}, hard_fail_on_parsing_errors: False
2022-09-12 23:21:54,148 [MainThread  ] [DEBUG]  No failed checks, or soft_fail is True and soft_fail_on and hard_fail_on are empty - returning 0

[gruebel]

From the logs I can only see that 0000-prometheuscustomresourcedefinitions-chart-c8e9cde4.k8s.yaml can't be read correctly. Can you share the first file, which is not scanned anymore? This one right 0012-consul-chart-c8e8de17.k8s.yaml?

[gruebel]

feel free to reach out again, if you have more info regarding this issue or other problems.