bdd-security icon indicating copy to clipboard operation
bdd-security copied to clipboard

Define site/app scanning

Open laminfaty opened this issue 10 years ago • 38 comments

Is there a way to tell bdd-security which is using owasp-zap not to scan the whole application/website

ex: Scan only http://mysite.com/thisapponly instead of http://mysite.com/

laminfaty avatar Feb 13 '15 20:02 laminfaty

Could you not change the baseUrl in the config file to point to the other location?

If not, then you can modify the runScanner method in the AppScanningSteps.java to scan http://mysite.com/thisapponly instead of the Config.getBaseUrl()

iriusrisk avatar Feb 14 '15 20:02 iriusrisk

Good stuff @continuumsecurity I noted that you add an ability to exclude some URLs. How about if someone want to exclude gazillions of URLs? I think the best approach is to be able to tell zap to spider and scan only an specific/unique URL. Let me know what you think.

laminfaty avatar Feb 18 '15 14:02 laminfaty

In exclude_urls.table, can you give an example of URL exclusion?

laminfaty avatar Feb 18 '15 15:02 laminfaty

I've just added that functionality today and I can't get it to work 100% in ZAP. E.g. if I use the ZAP API directly through the browser and tell it to ignore .*user.* Then do a scan of the entire site, I can see that some of the alerts were found in a url like: http://localhost/user/list/something But in the Active Scan tab of ZAP, no urls with .*user.* are listed. So it looks like it does scan those URLs, but then lies about it. Will have to double check this and discuss with the zap team.

iriusrisk avatar Feb 18 '15 15:02 iriusrisk

I have opened a similar issue few days ago. Maybe you can follow up and see what they are going to day.

http://code.google.com/p/zaproxy/issues/detail?id=1527&can=4

laminfaty avatar Feb 18 '15 18:02 laminfaty

I've not used the command line, but that is definitely possible in the API: http://zap/UI/ascan/action/scan/

iriusrisk avatar Feb 18 '15 19:02 iriusrisk

Cool. So, what is the plan going forward?

laminfaty avatar Feb 18 '15 19:02 laminfaty

Next iteration will have the URLs to scan in a table, similar to how it's done with the exclude_urls.table and how it's done in the spider in the navigate_app.story No ETA on it though.

iriusrisk avatar Feb 18 '15 20:02 iriusrisk

So, basically the URL to scan file will be dependent to the config file. Also in near future, we need to think about why to include some heavy fuzzing tests.

laminfaty avatar Feb 18 '15 20:02 laminfaty

The scanning scenarios will look like this:

Scenario: The application should not contain SQL injection vulnerabilities
Meta: @id scan_sql_injection
GivenStories: navigate_app.story
Given a scanner with all policies disabled
And the URL regular expressions listed in the file: tables/exclude_urls.table are excluded from the scanner
And the SQL-Injection policy is enabled
And the attack strength is set to High
And the alert threshold is set to Medium
When the scanner is run against the urls in: tables/include_urls.table
And false positives described in: tables/false_positives.table are removed
Then no Medium or higher risk vulnerabilities should be present

iriusrisk avatar Feb 18 '15 20:02 iriusrisk

ZAP doesn't expose its fuzzer in the API yet. Were you thinking of using another fuzzing tool?

iriusrisk avatar Feb 18 '15 20:02 iriusrisk

@continuumsecurity The above scenario looks very good. I use Kali a lot and I think it has ton of fuzzing tools. I will try to pick a good fuzzing tool. Do you have any specific criteria for fuzzing tool?

laminfaty avatar Feb 18 '15 20:02 laminfaty

Something popular that other users are likely to use :)

iriusrisk avatar Feb 18 '15 20:02 iriusrisk

Sounds good. When do you think these new functionalities will be done? Also did you take a look at the other issue that I open?

laminfaty avatar Feb 18 '15 20:02 laminfaty

@continuumsecurity I just tested the exclude-urls and it was ok, however, it excludes everything. So I think you need to add the include-urls functionality.

laminfaty avatar Feb 22 '15 00:02 laminfaty

@continuumsecurity This URL below is not part of the exclude-urls (regex), however it was filtered.

117813 [pool-1-thread-1] INFO org.zaproxy.zap.spider.SpiderController - URI: http://mywebsite/scanthisapp was filtered by a filter with reason: USER_RULES

laminfaty avatar Feb 22 '15 15:02 laminfaty

what regex are you using?

iriusrisk avatar Feb 22 '15 15:02 iriusrisk

Here is my exclude-urls table as you see these are my excluded urls.

|regex | |.scanthisapp.| |.my-login.|

laminfaty avatar Feb 22 '15 15:02 laminfaty

@continuumsecurity Any update or progress on the include-urls functionality to fixing owasp-zap scan.

laminfaty avatar Feb 23 '15 13:02 laminfaty

Earliest would be next week. The regex's in the table above don't look right. They should probably be:

    |regex                            |
    |.*scanthisapp.*             |
    |.*my-login.*                  |

iriusrisk avatar Feb 23 '15 13:02 iriusrisk

@continuumsecurity My regex has pipe and asterisk. You mean to tell me that no asterisk or pipe. Because it is markdown that's why the asterisk is not showing.

laminfaty avatar Feb 23 '15 13:02 laminfaty

Ah, gotcha, fixed my comment as well if someone else stumbles on this issue.

iriusrisk avatar Feb 23 '15 13:02 iriusrisk

So to clearify the regex will have pipe and asterisk. Coorect me if I am wrong. I think my regex is fine. The issue is zap filtering everything. Include-urls should fix this issue, I hope.

laminfaty avatar Feb 23 '15 13:02 laminfaty

Yes pipe and asterisk. I don't see why zap is filtering everything. Could you try doing the exact same thing but without the BDD-Sec framework. I.e.

  1. configure browser to use zap
  2. browse to target site and navigate around
  3. browse to http://zap
  • this will take you to the API
  • navigate to the ascan exclude urls option
  • add your regex
    1. go back to ZAP and choose to scan the site.

iriusrisk avatar Feb 23 '15 13:02 iriusrisk

Ok I will explore that.

laminfaty avatar Feb 23 '15 13:02 laminfaty

@continuumsecurity After deep analysis of zap proxy scanner, I find out that using a regex (/.someurl.). Basically, a forward slash, dot, and asterisk in front around the excluded url on API works (404 error when scanning the excluded url). I am still testing it and I think we can get it work if we use exclude-urls and include-urls.

laminfaty avatar Feb 23 '15 20:02 laminfaty

@continuumsecurity I hope you are doing well. I did some experimentations with Zap and find out that the best approach to get a specific URL scanned by Zap is to use: excludeFromScan (regex_) form the api http://zap.UI/spider scan (url_) from the api http://zap.UI/spider. This points to the URL that you wish to scan.

I tested it local and it works for me.

laminfaty avatar Mar 10 '15 14:03 laminfaty

Thanks @laminfaty, I haven't have time to add this functionality yet but I'm aware that it is still outstanding!

iriusrisk avatar Mar 22 '15 15:03 iriusrisk

You are welcome @continuumsecurity. If you need assistance please let me know. I am very excited and looking forward for that new functionality.

laminfaty avatar Mar 23 '15 17:03 laminfaty

@continuumsecurity, Any update on the include_urls functionality?

lfatty avatar Apr 13 '15 23:04 lfatty