detect-secrets
detect-secrets copied to clipboard
Yelp
AzureKeyDetector plugin (updated from AzureStorageKeyDetector)
I updated AzureStorageKeyDetector to AzureKeyDetector.
AzureKeyDetector supports various Azure services in addtion to Azure Storage (supported by AzureStorageKeyDetector).
Here are supported Azure services:
- Azure Storage
- Azure SQL Database
- Azure Database for PostgreSQL
- Azure Database for MySQL
- Azure Database for MariaDB
- Azure Cache for Redis
- Azure Cosmos DB
- Azure Synapse Analytics
- Azure Service Bus
- Azure Event Hubs
- Azure IoT Hub
- Azure Monitor
- Azure Functions
- Azure Web PubSub
- Azure SignalR Service
azure.py has RegEx expressions for various key/connection string format of various Azure services.
azure_test.py has corresponding payload/should_flag pairs for RegEx espressions/
azure_test.py runs succesfully.
$ python -m pytest tests/plugins/azure_test.py
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-0.13.1
rootdir: /mnt/c/repos/detect-secrets
collected 27 items
tests/plugins/azure_test.py ........................... [100%]
================================================================================================ 27 passed in 4.85s =================================================================================================
However, when I scan azure_test.py, detect-secrets detects 6 secrets only. I think it should detec 27 secrets. I don't know the reason why 21 secrets are not detected. I'd like to find the reason in this thread and fix it (if needed).
$ python3 -m detect_secrets scan tests/plugins/azure_test.py
{
"version": "1.3.0",
"plugins_used": [
{
"name": "ArtifactoryDetector"
},
{
"name": "AWSKeyDetector"
},
{
"name": "AzureKeyDetector"
},
{
"name": "Base64HighEntropyString",
"limit": 4.5
},
{
"name": "BasicAuthDetector"
},
{
"name": "CloudantDetector"
},
{
"name": "GitHubTokenDetector"
},
{
"name": "HexHighEntropyString",
"limit": 3.0
},
{
"name": "IbmCloudIamDetector"
},
{
"name": "IbmCosHmacDetector"
},
{
"name": "JwtTokenDetector"
},
{
"name": "KeywordDetector",
"keyword_exclude": ""
},
{
"name": "MailchimpDetector"
},
{
"name": "NpmDetector"
},
{
"name": "PrivateKeyDetector"
},
{
"name": "SendGridDetector"
},
{
"name": "SlackDetector"
},
{
"name": "SoftlayerDetector"
},
{
"name": "SquareOAuthDetector"
},
{
"name": "StripeDetector"
},
{
"name": "TwilioKeyDetector"
}
],
"filters_used": [
{
"path": "detect_secrets.filters.allowlist.is_line_allowlisted"
},
{
"path": "detect_secrets.filters.common.is_ignored_due_to_verification_policies",
"min_level": 2
},
{
"path": "detect_secrets.filters.gibberish.should_exclude_secret",
"limit": 3.7
},
{
"path": "detect_secrets.filters.heuristic.is_indirect_reference"
},
{
"path": "detect_secrets.filters.heuristic.is_likely_id_string"
},
{
"path": "detect_secrets.filters.heuristic.is_lock_file"
},
{
"path": "detect_secrets.filters.heuristic.is_not_alphanumeric_string"
},
{
"path": "detect_secrets.filters.heuristic.is_potential_uuid"
},
{
"path": "detect_secrets.filters.heuristic.is_prefixed_with_dollar_sign"
},
{
"path": "detect_secrets.filters.heuristic.is_sequential_string"
},
{
"path": "detect_secrets.filters.heuristic.is_swagger_file"
},
{
"path": "detect_secrets.filters.heuristic.is_templated_secret"
}
],
"results": {
"tests/plugins/azure_test.py": [
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "43cd99b89520080a5e7c1349235765d70ef15274",
"is_verified": false,
"line_number": 12
},
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "fd4eebe5e7bdb850e8e6471a4186f506324687f0",
"is_verified": false,
"line_number": 28
},
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "b27ea2fb585a210fbb607a675b71abe228db1c7d",
"is_verified": false,
"line_number": 60
},
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "7b162df4c3e8682cd7d6e6815376f4ee6903e41a",
"is_verified": false,
"line_number": 68
},
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "8ca69981e32198203d53f5a9dfcad52aa9846381",
"is_verified": false,
"line_number": 72
},
{
"type": "Azure keys/connection strings",
"filename": "tests/plugins/azure_test.py",
"hashed_secret": "e7722ae3eaa895362ea52b89e5a4d3b6d5440ce2",
"is_verified": false,
"line_number": 108
}
]
},
"generated_at": "2022-08-05T03:19:07Z"
}
Hi @satonaoki, thank you for opening this PR. I appreciate your work on improving our Azure keys detector.
~I ran our tests and it looks like they're failing, so I encourage you to take a look at those before we review your code. Let me know if you have any questions.~
I just merged #598, which fixed the probable cause why tests are failing. I suggest you update your branch and we'll run tests again.
Hi @lorenzodb1
I rebased my branch from Yelp:master.
The reason of test failure is that the old plugin name AzureStorageKeyDetector is included in .secrets.baseline. I updated it to AzureKeyDetector. Now tests succeeded.
$ python -m pytest tests
...
====================================================================================== warnings summary =======================================================================================
tests/core/baseline_test.py::TestCreate::test_error_when_getting_git_tracked_files
/mnt/c/repos/detect-secrets/detect_secrets/core/scan.py:74: UserWarning: Did not detect git repository. Try scanning all files instead.
log.warning('Did not detect git repository. Try scanning all files instead.')
tests/core/usage/filters_usage_test.py::test_disable_filter
/mnt/c/repos/detect-secrets/detect_secrets/core/usage/filters.py:192: UserWarning: Redundant --disable-filter "blah"
log.warning(f'Redundant --disable-filter "{name}"')
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================== 972 passed, 1 skipped, 6 xfailed, 2 warnings in 265.23s (0:04:25) ==============================================================
@satonaoki Hello. It seems the CI is still failing. Can you checkout the errors and fix them?
@jpdakran I added pragma allowlist comment to resolve the CI issue. Could you approve the workflow?
1 workflow awaiting approval First-time contributors need a maintainer to approve running workflows. [Learn more.]
@satonaoki looks like some checks are still failing. I encourage you to take a look at those.
@lorenzodb1 "W291 trailing whitespace" error fixed. Could you approve the workflow?
detect_secrets/plugins/azure.py:78:94: W291 trailing whitespace detect_secrets/plugins/azure.py:96:79: W291 trailing whitespace detect_secrets/plugins/azure.py:97:92: W291 trailing whitespace
Hmm. CI still has these two errors.
But it seems that azure.py has no issues for add-trailing-comma:
https://github.com/asottile/add-trailing-comma/blob/main/README.md
I will run CI in my forked repo and take a close look at it.
Add trailing commas......................................................Failed
- hook id: add-trailing-comma
- exit code: 1
- files were modified by this hook
Rewriting detect_secrets/plugins/azure.py
autopep8.................................................................Failed
- hook id: autopep8
- files were modified by this hook