findcdn icon indicating copy to clipboard operation
findcdn copied to clipboard

Handle invalid domains more gracefully

Open S4lt5 opened this issue 3 years ago โ€ข 3 comments

๐Ÿ—ฃ Description

As per #42, invalid domains should be handled gracefully.

These are checked for in main() before running the executor, and put in a separate "invalid domains" bucket.

Also, as part of code cleanup, the domain class was moved to a separate python file.

There is a LOT to do and a lot of redundant call chains to be removed, but each journey starts with a single step!

๐Ÿ’ญ Motivation and context

๐Ÿงช Testing

Added unit tests for valid/invalid cdnCheck() and main() calls with valid/invalid domains

Sample output

โฏ findcdn list "*.domain.com"
0 Domains Validated.
1 domain(s) rejected, listed below:
	*.domain.com
0it [00:00, ?it/s]
{
    "date": "11/21/2022, 11:44:09",
    "cdn_count": "0",
    "domains": {},
    "invalid_domains": [
        "*.domain.com"
    ]
}
Domain processing completed.
0 domains had CDN's out of 1.
1 Domains Validated.
2 domain(s) rejected, listed below:
	*.foo.bar
	wildcard domain
[Pending: 0 jobs]==[Threads: 1]: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1/1 [00:01<00:00,  1.26s/it]
{
    "date": "11/21/2022, 11:44:53",
    "cdn_count": "1",
    "domains": {
        "cisa.gov": {
            "IP": "'104.117.51.217'",
            "cdns": "'.edgekey.net', '.akamaitechnologies.fr'",
            "cdns_by_names": "'Akamai', 'Akamai'"
        }
    },
    "invalid_domains": [
        "*.foo.bar",
        "wildcard domain"
    ]
}
Domain processing completed.
1 domains had CDN's out of 3.

โœ… Pre-approval checklist

  • [ ] This PR has an informative and human-readable title.
  • [ ] Changes are limited to a single goal - eschew scope creep!
  • [ ] All future TODOs are captured in issues, which are referenced in code comments.
  • [ ] All relevant type-of-change labels have been added.
  • [ ] I have read the CONTRIBUTING document.
  • [ ] These code changes follow cisagov code standards.
  • [ ] All relevant repo and/or project documentation has been updated to reflect the changes in this PR.
  • [ ] Tests have been added and/or modified to cover the changes in this PR.
  • [ ] All new and existing tests pass.

โœ… Pre-merge checklist

  • [ ] Revert dependencies to default branches.
  • [ ] Finalize version.

โœ… Post-merge checklist

  • [ ] Add a tag or create a release.

S4lt5 avatar Nov 21 '22 17:11 S4lt5

This pull request introduces 1 alert when merging b3b320d1d889fcb73b5187ee4877250eb5d70e43 into 7bbf50e4a1cf1f32999c1abdcabc064d8ebf53f8 - view on LGTM.com

new alerts:

  • 1 for Unused import

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down โป completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine :gear: that powers LGTM.com. For more information, please check out our post on the GitHub blog.

lgtm-com[bot] avatar Nov 21 '22 17:11 lgtm-com[bot]

I think those "test_list_broken/est_file_broken" tests that fail, were failing before. I actually don't even understand the tests to be honest.

I'll double check with the other branches.

S4lt5 avatar Nov 23 '22 22:11 S4lt5

Good point. The tests should probably get a good review as well while I am on this.

Pascal-0x90 avatar Nov 24 '22 02:11 Pascal-0x90