tartufo Handle the case where exclude-signatures is a list of strings

Handle the case where exclude-signatures is a list of strings

Open jhall1-godaddy opened this issue 1 year ago • 5 comments

To help us get this pull request reviewed and merged quickly, please be sure to include the following items:

[x] Tests (if applicable)
[ ] Documentation (if applicable)
[x] Changelog entry
[x] A full explanation here in the PR description of the work done

PR Type

What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Tests
[ ] Other

Backward Compatibility

Is this change backward compatible with the most recently released version? Does it introduce changes which might change the user experience in any way? Does it alter the API in any way?

[x] Yes (backward compatible)
[ ] No (breaking changes)

Issue Linking

Closes #371 Closes #373

What's new?

I added an upwrap function to handle the case where exclude-signatures is a list of strings. I also added additional error handling for the case when there is no config file/no tool.tartufo section.

tomlkit 0.7.2, the current version in use by tartufo, had a bug where values would not update properly. I created a minimal reproducible example of that:

import tomlkit

toml_content = """
items = [
    "a",
    "c"
]
"""

data = tomlkit.loads(toml_content)
for i, item in enumerate(data["items"]):
    if item == "c":
        data["items"][i] = "b"

print(data["items"])
print([*data["items"]])

Result:

['a', 'c']
['a', 'b']

Expected result:

['a', 'b']
['a', 'b']

Based on this I updated tomlkit to the latest version 0.11.1 and that issue is now resolved. I added a couple more tests to handle the new cases.

Jul 21 '22 02:07 jhall1-godaddy

Turns out the docker build was failing because poetry decided to delete half the hashes from the lockfile while updating tomlkit. Resolved now.

Jul 21 '22 14:07 jhall1-godaddy

There is a new issue that was experienced relating to trying to pass bytes to click.echo. I haven't fully determined why this happens, but changing from a bytes() call to a str() call should resolve it. I will confirm that later.

Here is the relevant part of the traceback:

  File ".../tartufo/util.py", line 265, in process_issues
    echo_result(options, scan, repo_path, output_dir)
  File ".../tartufo/util.py", line 107, in echo_result
    click.echo(bytes(issue))
  File ".../click/utils.py", line 299, in echo
    file.write(out)  # type: ignore
TypeError: string argument expected, got 'bytes'

Jul 21 '22 16:07 jhall1-godaddy

Are you referring to the write_updated_signatures function?

If so, I did it to prevent opening the file twice, which felt inefficient.

Jul 21 '22 19:07 jhall1-godaddy

Ok this PR now addresses both the issue where exclude-signatures is a list of strings, and also where click.echo fails as described in #373

Am I missing anything? Should we try to gracefully handle different config misconfigurations, like if it's a list of dictionary's with no signature key, or any other potentially invalid configuration?

I feel like it might be best to try and prevent a traceback from hitting the end user, but it may obscure potentially useful information in the traceback. I'll defer to your feedback on whether that should be added.

Jul 22 '22 00:07 jhall1-godaddy

I have also noticed an issue where if duplicates are removed from the config, the final closing bracket is placed on the preceding line which can cause issues if that preceding line is only a comment after removing duplicates.

It's a pretty niche bug, but I know of at least 1 repository personally that would be affected. The end user's fix is to just drop the closing bracket to a newline - super simple - but a bug nonetheless.

A tracking issue has been created on the tomlkit repo here.

Jul 22 '22 13:07 jhall1-godaddy

Kudos, SonarCloud Quality Gate passed!