humble

A humble and fast HTTP Response Header Security Analyzer



"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Table of contents

Features
Screenshots
Installation & Update
Usage
Advanced Usage
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

:heavy_check_mark: 14 checks of missing HTTP response headers.
:heavy_check_mark: 1123 checks of fingerprinting through HTTP response headers.
:heavy_check_mark: 108 checks of deprecated HTTP response headers/protocols or with insecure/wrong values.
:heavy_check_mark: SSL/TLS checks: requires the wonderful https://testssl.sh/ and Linux/Unix OS.
:heavy_check_mark: Browser support references for enabled HTTP security headers.
:heavy_check_mark: Two types of analysis: brief and detailed, along with HTTP response headers.
:heavy_check_mark: Can exclude specific HTTP response headers from the analysis.
:heavy_check_mark: Can export each analysis to CSV, HTML5, JSON, PDF 1.4 and TXT (and in the PATH of your choice).
:heavy_check_mark: Each detailed analysis may include up to dozens of official links, references and technical articles.
:heavy_check_mark: l10n: can display each analysis, the messages and almost all errors in English or Spanish.
:heavy_check_mark: Saves each analysis, showing at the end the improvements or deficiencies in relation to the last one.
:heavy_check_mark: Can display analysis statistics: either against a specific URL or all of them.
:heavy_check_mark: Can display fingerprint statistics: either against a specific term or the Top 20.
:heavy_check_mark: Code reviewed via Bandit, Flake8, pyinstrument, SonarLint, Sourcery and vermin.
:heavy_check_mark: Tested (one by one) on thousands of URLs.
:heavy_check_mark: Tested on Kali Linux 2021.1, macOS 14.2.1 and Windows 10 20H2.
:heavy_check_mark: Almost all the code under one of the most permissive licenses: MIT.
:heavy_check_mark: Regularly updated.
:heavy_check_mark: Featured on OWASP, Kali Linux, Artemis (CERT Polska), DefectDojo and HackTricks!.
:heavy_check_mark: Developed entirely in my spare time, no strings attached: feel free to try it out and integrate it into your projects!.
:heavy_check_mark: And with the approval of several AI :smile:!.

Screenshots

.: (Windows) - Brief analysis.

Options used: -f -g -p -U -s --hints

Installation & Update

[!NOTE] Python 3.9 or higher is required.

# Install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...
(macOS) https://www.python.org/downloads/macos/

# Install Git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux
(macOS) https://git-scm.com/download/mac

# Clone this Git Repository
$ git clone https://github.com/rfc-st/humble.git

# Change the working directory to 'humble'
$ cd humble

# Install the required dependencies
$ pip3 install -r requirements.txt

# (Recommended) Check for updates weekly, inside 'humble' directory
$ git pull

# Or download the latest release, every four to five weeks
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py
(macOS)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-df] [-e [TESTSSL_PATH]] [-f [FINGERPRINT_TERM]] [-g] [-l {es}] [-o {csv,html,json,pdf,txt}] [-op OUTPUT_PATH] [-r]
                 [-s [SKIPPED_HEADERS ...]] [-u URL] [-ua USER_AGENT] [-v]

'humble' (HTTP Headers Analyzer) | https://github.com/rfc-st/humble | v.2024-04-28

options:
  -h, --help                  show this help message and exit
  -a                          Shows statistics of the performed analysis; will be global if the '-u' parameter is omitted
  -b                          Shows overall findings; if this parameter is omitted detailed ones will be shown
  -df                         Do not follow redirects; if this parameter is omitted the last redirection will be the one analyzed
  -e [TESTSSL_PATH]           Shows TLS/SSL checks; requires the 'TESTSSL_PATH' of https://testssl.sh/ and Linux/Unix OS
  -f [FINGERPRINT_TERM]       Shows fingerprint statistics; will be the Top 20 if 'FINGERPRINT_TERM', e.g. 'Google', is omitted
  -g                          Shows guidelines for enabling security HTTP response headers on popular servers/services
  -l {es}                     The language for displaying analysis, errors and messages; will be in English if this parameter is omitted
  -o {csv,html,json,pdf,txt}  Exports analysis to 'scheme_host_port_yyyymmdd.ext' file; csv/json files will contain a brief analysis
  -op OUTPUT_PATH             Exports analysis to 'OUTPUT_PATH'; if this parameter is omitted the PATH of 'humble.py' will be used
  -r                          Shows HTTP response headers and a detailed analysis; '-b' parameter will take priority
  -s [SKIPPED_HEADERS ...]    Skip analysis of HTTP response headers specified in 'SKIPPED_HEADERS' (separated by spaces)
  -u URL                      Scheme, host and port to analyze. E.g. https://google.com
  -ua USER_AGENT              User-Agent ID from 'additional/user_agents.txt' to use. '0' will show all and '1' is the default
  -v, --version               Checks for updates at https://github.com/rfc-st/humble

examples:
  -a -l es                    Shows statistics (in Spanish) of the analysis performed against all URLs
  -f Google                   Shows HTTP fingerprint headers related to the term 'Google'
  -u URL -a                   Shows statistics of the analysis performed against the URL
  -u URL -b                   Analyzes the URL and reports overall findings
  -u URL -b -o csv            Analyzes the URL and exports overall findings to CSV
  -u URL -l es                Analyzes the URL and reports (in Spanish) detailed findings
  -u URL -o pdf               Analyzes the URL and exports detailed findings to PDF
  -u URL -r                   Analyzes the URL and reports detailed findings along with HTTP response headers
  -u URL -s ETag NEL          Analyzes the URL and skips checks associated with 'ETag' and 'NEL' HTTP response headers
  -u URL -ua 4                Analyzes the URL using the fourth User-Agent of 'additional/user_agents.txt'

Advanced Usage

.: (Linux) - Show only the analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'

.: (Windows) - Show only the analysis summary, in Spanish. PowerShell >= 7 required.

$ py humble.py -u https://www.spacex.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

.: (Linux) - Show only the URL, date and analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

.: (Linux) - Show only the deprecated headers/protocols and insecure values.

$ python3 humble.py -u https://www.spacex.com | sed -n '/\[3/,/^\[4/ { /^\[4/!p }' | sed '$d' | sed $'1i \n'

.: (Linux) - Check for HTTP client errors (4XX).

$ python3 humble.py -u https://my.prelude.software/demo/index.pl | grep -A1 -B5 'Note : \|Nota : ' --color=never

.: (Linux) - Analyze multiple URLs and save the results as PDFs.

$ datasets=('https://facebook.com' 'https://github.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done

Checks: Missing Headers

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

[!NOTE] humble tries to be strict: both in checking HTTP response headers and their values; some of these headers may be experimental and you may not agree with all the results after analysis.

And that's OK! :smiley:; you should never blindly trust the results of security tools: there should be further work to decide whether the risk is non-existent, potential or real depending on the analyzed URL (its exposure, environment, etc).

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

• Amazon Web Services
• Apache HTTP Server
• Cloudflare
• LiteSpeed Web Server
• Microsoft Internet Information Services
• Nginx
• Node.js
• WordPress

To-Do

• [ ] Add more Header/Value checks (only security-oriented)
• [ ] A new detailed analysis of all CSP directives/values (W3C Level 2 & 3)
• [ ] Google Style Python Docstrings and documentation via Sphinx

Further reading

https://caniuse.com/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://github.com/search?q=http+headers+analyze
https://github.com/search?q=http+headers+secure
https://github.com/search?q=http+headers+security
https://owasp.org/www-project-secure-headers/
https://securityheaders.com/
https://scotthelme.co.uk/
https://webtechsurvey.com/common-response-headers
https://www.w3.org

Contribute

• Report a Bug.
• Create a Feature request.
• Report a Security Vulnerability.
• Send me your suggestions: [email protected]
• Or use that email to tell me about integrations of this tool in others!
• And to recommend me a good Blues! :sunglasses:

Thanks for downloading 'humble', for trying it and for your time!.

Acknowledgements

• Bandit, colorama, Flake8, fpdf2, pyinstrument, requests, SonarLint, Sourcery, tldextract and Vermin authors/teams: you rock :metal:!.
• Aniket Navlur for this gem.
• Azathothas for reporting this bug.
• bulaktm for this suggestion.
• David for believing in the usefulness of this tool.
• Eduardo for the first Demo and the example "(Linux) - Analyze multiple URLs and save the results as PDFs".
• gl4nce for this suggestion.
• İDRİS BUDAK for reporting the need to this check.
• manuel-sommer for this, this and this!.
• stanley101music for this, this and this!.
• n3bojs4, ehlewis and dkadev for this and this.
• kazet for this suggestion.
• Julio for testing on macOS.

License

MIT © 2020-2024 Rafa 'Bluesman' Faura ([email protected])
Original Creator - Rafa 'Bluesman' Faura ([email protected])