wstg
wstg copied to clipboard
OWASP
Properly order document sections
What's the issue?
The WSTG has a legacy layout that doesn't account for some new sections and tests. I think section 4 itself has become an unnecessary layer, and the test types can be surfaced by a step for easier navigation of the guide. Article titles would benefit from the removal of redundant words, like "Testing of."
How do we solve it?
Reorder sections into a more sensible and futureproof format. I propose alphabetical by major sections, then minor articles. This issue is intended to invite feedback and work on the ordering collaboratively.
For ease of work, I will submit the proposed ordering as a TOC file (#254), which if approved, should replace documet/README.md
TODO:
- [x] Agree on chapters layout
- [ ] Assign section numbers
- [ ] Add test identifiers
- [ ] Link-ify section names once reorganizing is complete
We need to take into consideration how other projects are setting this up, which could allow for an easier process to migrate if something was to happen. As discussions are erupting as well with the ASVS team in order to make it easier for the mapping between the 2 projects, I think we need to properly discuss this and visualize a structure that could allow us to easily map ASVS to WSTG, if it was to help. @kingthorin and @rejahrehim your input is highly required. V is already in this conversation 😄
Hmmm I'm not entirely sure about the alpha ordering.
Dropping chapter identifiers could be fine, but then we also need to drop figure numbering (which is probably fine, but we should agree and be sure).
I think dropping test identifiers is a mistake though (as that's the most simple mechanism by which projects can cross-reference each other's content) and will cause confusion as far as renumbering existing scenarios to match alpha sorting [perhaps we accept that, but we shouldn't do it without thought and perhaps community input (twitter poll??) ].
Dropping chapter identifiers could be fine, but then we also need to drop figure numbering (which is probably fine, but we should agree and be sure).
I think dropping test identifiers is a mistake...
I didn’t think we’d drop either of those; sorry, I scattered my info between the issue and the PR. I’ve updated my issue comment to include everything now. For clarity, the chapter and section numbers should be added in after we’ve settled the ordering (since they will have grown) and same for the test IDs.
Syncing with ASVS is definitely a good idea, although I'm worried it won't make sense in a lot of cases. Examples
- ASVS v1.1, v1.8 deals with app design and (human) management process don't map to anything in scope of WSTG
- Any input sanitization or serialization checks could apply to multiple areas (ASVS v2,x ASVS v4.x) because of course failure of any user input verification can break authentication, but input verifications also match ASVS v4.x. I suppose what I'm trying to say here is we could have one-to-many or many-to-many mappings of ASVS expectations and WSTG sections.
- ASVS v11.x talks about the general flow of business logic in the app. It sounds like something at the high level that you can test for, but to find it, a human would have to piece together different failures at the lower level to know that the app's design may be wrong with regards to security. I'm not sure if that doesn't go in WSTG at all or if it ends up as a one-to-many mapping.
Those are valid points. Requirements can map to multiple test scenarios, the opposite can happen as well. Requirements are the lowest level in the chain, so I'd expect more from that side. No need to mimic the structure. The structure won't bother anyone. It could assist, and you raised important points to it. I think we'll be able to finalize how this links in 2 weeks if we keep these discussions up 😄
Besides the possibility of sections in WSTG or ASVS moving around in the future, I don't think attempting to match up section numbers with the other guides would be helpful. I worry it will be more confusing than convenient, to @patrickceg's point.
I think the most helpful we could be for creating mappings is just to mention related sections of ASVS in the text of the WSTG, where appropriate.
I think the most helpful we could be for creating mappings is just to mention related sections of ASVS in the text of the WSTG, where appropriate.
I was thinking about that, and my idea was another project that just maintains a graph (the discrete mathematics graph https://en.wikibooks.org/wiki/Discrete_Mathematics/Graph_theory#Introduction ) that maps various section or article IDs (OWASP WSTG, OWAS MSTG, OWASP ASVS, Cheat Sheet, OWASP Top 10, CWE, SANS top 25, whatever else comes out next week). The linker project would then have a bunch of interfaces to answer queries including:
- "Which of the OWASP ASVS rules do I check off if I address the top 10?"
- "I want to be test for [insert test of ASVS]: Which mobile test and web app test cases do I write?"
I reiterate that's a completely separate project: I'm sure even keeping something like that in sync will be a job on its own.
It is a project currently being studied @patrickceg 😄 I will DM you about it.
@victoriadrake let's say we're moving the current structure numbering, what would we use? I am looking into a friendlier way than having <section-number>_<name><test-id>. What would be a neat way to do this? I would love to keep the test structure though, it helps to have unique URLs for tests for references, instead of having 10 tests under one markdown file.
The layout has been agreed upon in: https://github.com/OWASP/wstg/pull/254 as follows:
WSTG Table of Contents
- README.md (Table of Contents)
- Foreword
- README.md (formerly 0_Foreword.md)
- Frontispiece
- README.md (formerly 1_Frontispiece.md)
- Introduction_to_the_Guide
- README.md (formerly 2_Introduction.md)
- Introduction_to_Web_Testing (formerly 4.1_Introduction_to_Web_Testing)
- The_OWASP_Testing_Framework (formerly 3_The_OWASP_Testing_Framework.md)
- Penetration_Testing_Methodologies (formerly 3.8_Penetration_Testing_Methodologies.md)
- README.md (formerly 4.1_Testing_Introduction_and_Objectives.md)
- Testing_Checklist (formerly 4.1.1_Testing_Checklist.md)
- Information_Gathering (formerly 4.2_Information_Gathering)
- README.md (Table of Contents - formerly 4.2_Testing_Information_Gathering.md)
- Application_Entry_Points (formerly 4.2.6_Identify_Application_Entry_Points_OTG-INFO-006.md)
- Application_Execution_Paths (formerly 4.2.7_Map_Execution_Paths_Through_Application_OTG-INFO-007.md)
- Application_Framework (formerly 4.2.8_Fingerprint_Web_Application_Framework_OTG-INFO-008.md)
- Applications_on_Webserver (formerly 4.2.4_Enumerate_Applications_on_Webserver_OTG-INFO-004.md)
- Fingerprint_Application (formerly 4.2.9_Fingerprint_Web_Application_OTG-INFO-009.md)
- Fingerprint_Web_Server (formerly 4.2.2_Fingerprint_Web_Server_OTG-INFO-002.md)
- Map_Application_Architecture (formerly 4.2.10_Map_Application_Architecture_OTG-INFO-010.md)
- Search_Engine_Discovery_Reconnaissance (formerly 4.2.1_Conduct_Search_Engine_Discovery_Reconnaissance_for_Information_Leakage_OTG-INFO-001.md)
- Webpage_Comments_and_Metadata (formerly 4.2.5_Review_Webpage_Comments_and_Metadata_for_Information_Leakage_OTG-INFO-005.md)
- Webserver_Metafiles (formerly 4.2.3_Review_Webserver_Metafiles_for_Information_Leakage_OTG-INFO-003.md)
- Testing
- Application_Programming_Interface_Testing (formerly Testing_for_APIs.md)
- Authentication_Testing (formerly 4.5_Authentication_Testing)
- README.md (Table of Contents - formerly 4.5_Testing_for_Authentication.md)
- Browser_Cache_Weaknesses (formerly 4.5.6_Testing_for_Browser_Cache_Weaknesses_OTG-AUTHN-006.md)
- Bypassing_Authentication_Schema (formerly 4.5.4_Testing_for_Bypassing_Authentication_Schema_OTG-AUTHN-004.md)
- Credentials_Transported_over_Encrypted_Channel (formerly 4.5.1_Testing_for_Credentials_Transported_over_an_Encrypted_Channel_OTG-AUTHN-001.md)
- Default_Credentials (formerly 4.5.2_Testing_for_Default_Credentials_OTG-AUTHN-002.md)
- Password_Change_or_Reset_Weakness (formerly 4.5.9_Testing_for_Weak_Password_Change_or_Reset_Functionalities_OTG-AUTHN-009.md)
- Password_Policy_Weakness (formerly 4.5.7_Testing_for_Weak_Password_Policy_OTG-AUTHN-007.md)
- Password_Remembering (formerly 4.5.5_Testing_for_Vulnerable_Remember_Password_OTG-AUTHN-005.md)
- Security_Question_Weakness (formerly 4.5.8_Testing_for_Weak_Security_Question_Answer_OTG-AUTHN-008.md)
- Weak_Lock_Out_Mechanism (formerly 4.5.3_Testing_for_Weak_Lock_Out_Mechanism_OTG-AUTHN-003.md)
- Weaker_Authentication_in_Alternative_Channel (formerly 4.5.10_Testing_for_Weaker_Authentication_in_Alternative_Channel_OTG-AUTHN-010.md)
- Authorization_Testing (formerly 4.6_Authorization_Testing)
- README.md (Table of Contents - formerly 4.6_Testing_for_Authorization.md)
- Bypassing_Authorization_Schema (formerly 4.6.2_Testing_for_Bypassing_Authorization_Schema_OTG-AUTHZ-002.md)
- README.md
- Horizontal_Bypassing (formerly Testing_for_Horizontal_Bypassing_Authorization_Schema_OTG-AUTHZ-002.md)
- Vertical_Bypassing (formerly Testing_for_Vertical_Bypassing_Authorization_Schema_OTG-AUTHZ-00X.md)
- Directory_Traversal (formerly 4.6.1_Testing_Directory_Traversal_File_Include_OTG-AUTHZ-001.md)
- Insecure_Direct_Object_Reference (formerly 4.6.4_Testing_for_Insecure_Direct_Object_References_OTG-AUTHZ-004.md)
- Privilege_Escalation (formerly 4.6.3_Testing_for_Privilege_Escalation_OTG-AUTHZ-003.md)
- Business_Logic_Testing (formerly 4.11_Business_Logic_Testing)
- README.md (Table of Contents - formerly 4.11_Testing_for_Business_Logic.md)
- Application_Misuse (formerly 4.11.7_Test_Defenses_Against_Application_Mis-use_OTG-BUSLOGIC-007.md)
- Circumvention_of_Work_Flows (formerly 4.11.6_Testing_for_the_Circumvention_of_Work_Flows_OTG-BUSLOGIC-006.md)
- Data_Validation (formerly 4.11.1_Test_Business_Logic_Data_Validation_OTG-BUSLOGIC-001.md)
- Integrity_Checks (formerly 4.11.3_Test_Integrity_Checks_OTG-BUSLOGIC-003.md)
- Limits_for_Function_Use (formerly 4.11.5_Test_Number_of_Times_a_Function_Can_Be_Used_Limits_OTG-BUSLOGIC-005.md)
- Process_Timing (formerly 4.11.4_Test_for_Process_Timing_OTG-BUSLOGIC-004.md)
- Request_Forgery (formerly 4.11.2_Test_Ability_to_Forge_Requests_OTG-BUSLOGIC-002.md)
- Upload_of_Malicious_Files (formerly 4.11.9_Test_Upload_of_Malicious_Files_OTG-BUSLOGIC-009.md)
- Upload_of_Unexpected_File_Type (formerly 4.11.8_Test_Upload_of_Unexpected_File_Types_OTG-BUSLOGIC-008.md)
- Client_Side_Testing (formerly 4.12_Client_Side_Testing)
- README.md (Table of Contents - formerly 4.12_Client_Side_Testing.md)
- Clickjacking (formerly 4.12.9_Testing_for_Clickjacking_OTG-CLIENT-009.md)
- Cross_Origin_Resource_Sharing (formerly 4.12.7_Testing_Cross_Origin_Resource_Sharing_OTG-CLIENT-007.md)
- Cross_Site_Flashing (formerly 4.12.8_Testing_for_Cross_Site_Flashing_OTG-CLIENT-008.md)
- Cross_Site_Script_Inclusion (formerly 4.12.13_Testing_for_Cross_Site_Script_Inclusion_OTG-CLIENT-013.md)
- CSS_Injection (formerly 4.12.5_Testing_for_CSS_Injection_OTG-CLIENT-005.md)
- DOM-based_Cross_Site_Scripting (formerly 4.12.1_Testing_for_DOM-based_Cross_Site_Scripting_OTG-CLIENT-001.md)
- HTML_Injection (formerly 4.12.3_Testing_for_HTML_Injection_OTG-CLIENT-003.md)
- JavaScript_Execution (formerly 4.12.2_Testing_for_JavaScript_Execution_OTG-CLIENT-002.md)
- Web_Messaging (formerly 4.12.11_Testing_Web_Messaging_OTG-CLIENT-011.md)
- Web_Storage (formerly 4.12.12_Testing_Web_Storage_OTG-CLIENT-012.md)
- Websockets (formerly 4.12.10_Testing_WebSockets_OTG-CLIENT-010.md)
- Resource_Manipulation (formerly 4.12.6_Testing_for_Client_Side_Resource_Manipulation_OTG-CLIENT-006.md)
- URL_Redirect (formerly 4.12.4_Testing_for_Client_Side_URL_Redirect_OTG-CLIENT-004.md)
- Configuration_and_Deployment_Management_Testing (formerly 4.3_Configuration_and_Deployment_Management_Testing)
- README.md (Table of Contents - formerly 4.3_Testing_for_Configuration_Management.md)
- Application_Platform_Configuration (formerly 4.3.2_Test_Application_Platform_Configuration_OTG-CONFIG-002.md)
- File_Permissions (formerly 4.3.9_Test_File_Permission_OTG-CONFIG-009.md)
- HTTP_Methods (formerly 4.3.6_Test_HTTP_Methods_OTG-CONFIG-006.md)
- HTTP_Strict_Transport_Security (formerly 4.3.7_Test_HTTP_Strict_Transport_Security_OTG-CONFIG-007.md)
- Infrastructure_and_Application_Admin_Interfaces (formerly 4.3.5_Enumerate_Infrastructure_and_Application_Admin_Interfaces_OTG-CONFIG-005.md)
- Network_Infrastructure_Configuration (formerly 4.3.1_Test_Network_Infrastructure_Configuration_OTG-CONFIG-001.md)
- Rich_Internet_Applications_Cross_Domain_Policy (formerly 4.3.8_Test_RIA_Cross_Domain_Policy_OTG-CONFIG-008.md)
- Sensitive_Information_in_File_Extensions_Handling (formerly 4.3.3_Test_File_Extensions_Handling_for_Sensitive_Information_OTG-CONFIG-003.md)
- Sensitive_Information_in_Forgotten_Files (formerly 4.3.4_Review_Old_Backup_and_Unreferenced_Files_for_Sensitive_Information_OTG-CONFIG-004.md)
- Subdomain_Takeover (formerly 4.3.10_Test_for_Subdomain_Takeover_OTG-CONFIG-010.md)
- Cryptography_and_Encryption_Testing (formerly 4.10_Testing_for_Weak_Cryptography)
- README.md (Table of Contents - formerly 4.10_Testing_for_Weak_Cryptography.md)
- Padding_Oracle (formerly 4.10.2_Testing_for_Padding_Oracle_OTG-CRYPST-002.md)
- Unencrypted_Channels (formerly 4.10.3_Testing_for_Sensitive_Information_Sent_via_Unencrypted_Channels_OTG-CRYPST-003.md)
- Weak_Cryptography (formerly 4.10.1_Testing_for_Weak_SSL_TLS_Ciphers_Insufficient_Transport_Layer_Protection_OTG-CRYPST-001.md)
- Weak_Encryption (formerly 4.10.4_Testing_for_Weak_Encryption_OTG-CRYPST-004.md)
- Error_Handling_Testing (formerly 4.9_Testing_for_Error_Handling)
- README.md (Table of Contents - formerly 4.9_Testing_for_Error_Handling.md)
- Error_Code (formerly 4.9.1_Testing_for_Error_Code_OTG-ERR-001.md)
- Stack_Traces (formerly 4.9.2_Testing_for_Stack_Traces_OTG-ERR-002.md)
- Identity_Management_Testing (formerly 4.4_Identity_Management_Testing)
- README.md (Table of Contents - formerly 4.4_Identity_Management_Testing.md)
- Account_Enumeration_and_Guessable_User_Account (formerly 4.4.4_Testing_for_Account_Enumeration_and_Guessable_User_Account_OTG-IDENT-004.md)
- Account_Provisioning_Process (formerly 4.4.3_Test_Account_Provisioning_Process_OTG-IDENT-003.md)
- Role_Definitions (formerly 4.4.1_Test_Role_Definitions_OTG-IDENT-001.md)
- User_Registration_Process (formerly 4.4.2_Test_User_Registration_Process_OTG-IDENT-002.md)
- Username_Policy (formerly 4.4.5_Testing_for_Weak_or_Unenforced_Username_Policy_OTG-IDENT-005.md)
- Input_Validation_Testing (formerly 4.8_Input_Validation_Testing)
- README.md (Table of Contents - formerly 4.8_Testing_for_Input_Validation.md)
- Buffer_Overflow (formerly 4.8.14_Testing_for_Buffer_Overflow_OTG-INPVAL-014.md)
- README.md
- Format_String (formerly 4.8.14.3_Testing_for_Format_String.md)
- Heap_Overflow (formerly 4.8.14.1_Testing_for_Heap_Overflow.md)
- Stack_Overflow (formerly 4.8.14.2_Testing_for_Stack_Overflow.md)
- Code_Injection (formerly 4.8.12_Testing_for_Code_Injection_OTG-INPVAL-012.md)
- README.md
- Local_File_Inclusion (formerly 4.8.12.1_Testing_for_Local_File_Inclusion.md)
- Remote_File_Inclusion(formerly 4.8.12.2_Testing_for_Remote_File_Inclusion.md)
- Command_Injection (formerly 4.8.13_Testing_for_Command_Injection_OTG-INPVAL-013.md)
- Host_Header_Injection (formerly 4.8.18_Testing_for_Host_Header_Injection_OTG-INPVAL-018.md)
- HTTP_Incoming_Requests (formerly 4.8.17_Testing_for_HTTP_Incoming_Requests_OTG-INPVAL-017.md)
- HTTP_Parameter_Pollution (formerly 4.8.4_Testing_for_HTTP_Parameter_Pollution_OTG-INPVAL-004.md)
- HTTP_Splitting_Smuggling (formerly 4.8.16_Testing_for_HTTP_Splitting_Smuggling_OTG-INPVAL-016.md)
- HTTP_Verb_Tampering (formerly 4.8.3_Testing_for_HTTP_Verb_Tampering_OTG-INPVAL-003.md)
- IMAP_SMTP_Injection (formerly 4.8.11_Testing_for_IMAP_SMTP_Injection_OTG-INPVAL-011.md)
- Incubated_Vulnerability (formerly 4.8.15_Testing_for_Incubated_Vulnerability_OTG-INPVAL-015.md)
- LDAP_Injection (formerly 4.8.6_Testing_for_LDAP_Injection_OTG-INPVAL-006.md)
- Object_Relational_Mapping_Injection (formerly 4.8.7_Testing_for_ORM_Injection_OTG-INPVAL-007.md)
- Reflected_Cross_Site_Scripting (formerly 4.8.1_Testing_for_Reflected_Cross_Site_Scripting_OTG-INPVAL-001.md)
- Server_Side_Includes_Injection (formerly 4.8.9_Testing_for_SSI_Injection_OTG-INPVAL-009.md)
- Server_Side_Template_Injection (formerly 4.8.19_Testing_for_Server_Side_Template_Injection_OTG-INPVAL-019.md)
- Stored_Cross_Site_Scripting (formerly 4.8.2_Testing_for_Stored_Cross_Site_Scripting_OTG-INPVAL-002.md)
- SQL_Injection (formerly 4.8.5_Testing_for_SQL_Injection_OTG-INPVAL-005.md)
- README.md
- Oracle (formerly 4.8.5.1_Testing_for_Oracle.md)
- MySQL (formerly 4.8.5.2_Testing_for_MySQL.md)
- MS_Access (formerly 4.8.5.5_Testing_for_MS_Access.md)
- NoSQL_Injection (formerly 4.8.5.6_Testing_for_NoSQL_Injection.md)
- PostgreSQL (formerly 4.8.5.4_OWASP_Backend_Security_Project_Testing_PostgreSQL.md)
- SQL_Server (formerly 4.8.5.3_Testing_for_SQL_Server.md)
- XML_Injection (formerly 4.8.8_Testing_for_XML_Injection_OTG-INPVAL-008.md)
- XPath_Injection (formerly 4.8.10_Testing_for_XPath_Injection_OTG-INPVAL-010.md)
- Server-Side_Request_Forgery_Testing (formerly Testing_for_Server-Side_Request_Forgery.md)
- Session_Management_Testing (formerly 4.7_Session_Management_Testing)
- README.md (Table of Contents - formerly 4.7_Testing_for_Session_Management.md)
- Cookies_Attributes (formerly 4.7.2_Testing_for_Cookies_Attributes_OTG-SESS-002.md)
- Cross_Site_Request_Forgery (formerly 4.7.5_Testing_for_CSRF_OTG-SESS-005.md)
- Exposed_Session_Variables (formerly 4.7.4_Testing_for_Exposed_Session_Variables_OTG-SESS-004.md)
- Logout_Functionality (formerly 4.7.6_Testing_for_Logout_Functionality_OTG-SESS-006.md)
- Session_Fixation (formerly 4.7.3_Testing_for_Session_Fixation_OTG-SESS-003.md)
- Session_Management_Schema (formerly 4.7.1_Testing_for_Session_Management_Schema_OTG-SESS-001.md)
- Session_Puzzling (formerly 4.7.8_Testing_for_Session_Puzzling_OTG-SESS-008.md)
- Session_Timeout (formerly 4.7.7_Test_Session_Timeout_OTG-SESS-007.md)
- Reporting (formerly 5_Reporting)
- README.md (formerly 5_Reporting.md)
- Appendix
- Appx.A_Testing_Tools.md
- Appx.B_Suggested_Reading.md
- Appx.C_Fuzz_Vectors.md
- Appx.D_Encoded_Injection.md
- Appx.E_Revision_History.md
- README.md
Please comment if you are still working on this issue, as it has been inactive for 30 days. To give everyone a chance to contribute, we are releasing it to new contributors.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Can we take this forward? We have to make changes in the build scripts accordingly. Need this to be done for Json generation too. @kingthorin @ThunderSon @victoriadrake
This one is a bit difficult since it introduces what one might call "breaking changes" as far as where things are located and how they're numbered/sequenced.
I'm personally fine with breaking things in a minor release, however, I understand if others are against that.
I'm on the camp of break things whenever the value to the project is worth it because versions and version control systems exist. Place a note on the README that the big section reorg happened between version X and Y. They can always go back by tags or by using the older PDF version.
As a user, I just expect everything to break every release: I've been like that since I started using Android devices... (😁 is there anything in tech that doesn't have breaking changes every time? I think the closest thing for me that doesn't break often on updates is Debian Linux).
It's similar to why you won't want your Dockerfile to reference ubuntu:latest unless you know you'll be in there fixing the build every so often. We have tags on this repository, so for the people who want to link to something less in flux, they can use the tag.
This one is a bit difficult since it introduces what one might call "breaking changes" as far as where things are located and how they're numbered/sequenced.
I'm personally fine with breaking things in a minor release, however, I understand if others are against that.
We are planning for the next big release (V5). Right ? So, We have to do this anyway.
We could delay longer and plan for that I suppose. That’d be fine with me.
Sure, we'd need to make a greater push to give v5 more value to its name :) Let's see what we can manage to do.