manual / automated authenticated crawling with project options (headless)

[ehsandeep]

Feature suggested by @parthmalhotra

Please describe your feature request:

New options to add for this feature:

HEADLESS:
   -ob, -open-browser               open chrome browser for manual browsing and crawling


PROJECT:
   -cp, -crawl-project          create/use project data for authenticated crawl
   -lp, -list-project           list previously stored project

Example runs:

katana -headless -open-browser -crawl-project test # open browser with blank url
katana -headless -open-browser -u https://hackerone.com -crawl-project h1 # open browser with https://hackerone.com as url to navigate
katana -headless -u https://hackerone.com -cp h1  # automated crawling from project data (session information)
katana -headless -cp h1  # automated crawling from project data (session information)

katana --list-project

/Users/geekboy/Github/katana/test1  Nov 29 15:13 20MB
/Users/geekboy/Github/katana/test2  Nov 29 15:13 10MB
/Users/geekboy/Github/katana/test3  Nov 29 15:13 100KB

Following headless options can be used along with -sc or -scp option (https://github.com/projectdiscovery/katana/pull/202) internally to create / reuse session information from the disk.

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir="test"

Note:

1. When --open-browser option is used, manually browsed information will be used as seed information to perform crawling in the background and displayed in the CLI output.
2. -ob and -sb can not be used together.
3. -sc and -scp can not be used together.
4. -cp option can be used to create / reuse session information when used with -ob option.
5. -cp option can be only used to reuse session information if not used with -ob option.
6. -cp option can be only used in headless mode as of now; planned to work with the standard mode in future.

[dogancanbakir]

Related, https://github.com/projectdiscovery/katana/issues/43