Refactoring of the BodySnifferThrottle.

[boocmp]

Resolves https://github.com/brave/brave-browser/issues/36532

Submitter Checklist:

• [ ] I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
• [ ] There is a ticket for my issue
• [ ] Used Github auto-closing keywords in the PR description above
• [ ] Wrote a good PR/commit description
• [ ] Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
• [ ] Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
• [ ] Checked the PR locally:
  • npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
  • npm run lint, npm run presubmit wiki, npm run gn_check, npm run tslint
• [ ] Ran git rebase master (if needed)

Reviewer Checklist:

• [ ] A security review is not needed, or a link to one is included in the PR description
• [ ] New files have MPL-2.0 license header
• [ ] Adequate test coverage exists to prevent regressions
• [ ] Major classes, functions and non-trivial code blocks are well-commented
• [ ] Changes in component dependencies are properly reflected in gn
• [ ] Code follows the style guide
• [ ] Test plan is specified in PR before merging

After-merge Checklist:

• [ ] The associated issue milestone is set to the smallest version that the changes has landed on
• [ ] All relevant documentation has been updated, for instance:
  • [ ] https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • [ ] https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • [ ] https://github.com/brave/brave-browser/wiki/Fingerprinting-Protections
  • [ ] https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • [ ] https://github.com/brave/brave-browser/wiki/Web-Compatibility-Exceptions-in-Brave
  • [ ] https://github.com/brave/brave-browser/wiki/QA-Guide
  • [ ] https://github.com/brave/brave-browser/wiki/P3A

Test Plan:

[iefremov]

pls rebase and fulfil the checklist

[iefremov]

@boocmp please complete the checklist

[iefremov]

@petemill FYI this was updated to include the AI throttle

[github-actions[bot]]

[puLL-Merge] - brave/brave-core@21792

Description

This PR adds a new BodySnifferThrottle that allows intercepting and modifying the response body for specific requests. It refactors existing code in speedreader, de_amp, and ai_chat components to use this new throttle.

Changes

Changes

• brave/browser/ai_chat/BUILD.gn: Adds a new browsertest file.
• brave/browser/ai_chat/ai_chat_browsertests.cc: Adds a new browsertest for YouTube navigations and AI chat interception.
• brave/browser/brave_content_browser_client.cc: Updates to use BodySnifferThrottle for de_amp and speedreader. Removes usage of DeAmpThrottle.
• brave/components/ai_chat/content/browser/page_content_fetcher.cc: Adds null checks.
• brave/components/ai_chat/renderer/BUILD.gn, brave/components/ai_chat/renderer/ai_chat_resource_sniffer.cc, brave/components/ai_chat/renderer/ai_chat_resource_sniffer.h: Refactors AI chat resource sniffing to use BodyHandler interface. Removes old AI chat resource sniffer throttle and url loader.
• brave/components/body_sniffer/*: Modifies BodySnifferThrottle to allow adding multiple BodyHandlers. Refactors BodySnifferURLLoader to work with the new BodyHandler interface.
• brave/components/de_amp/*: Refactors de-amp to use BodyHandler instead of a custom throttle and url loader.
• brave/components/speedreader/*: Refactors speedreader to use BodyHandler instead of a custom throttle and url loader. Adds a new BodyProducer for serving distilled pages.

Security Hotspots

• browser/ai_chat/ai_chat_browsertests.cc: The new browsertest starts an HTTPS server without certificate verification. This is okay for testing but should not be done in production code.
• components/ai_chat/renderer/ai_chat_resource_sniffer.cc, components/de_amp/browser/de_amp_body_handler.cc, components/speedreader/speedreader_body_distiller.cc, components/speedreader/speedreader_distilled_page_producer.cc: The refactored handlers run JavaScript provided in the page. This could potentially be abused for XSS if the JavaScript modifies page contents in an unsafe way. The code should validate and sanitize any dynamic content.
• components/body_sniffer/*: The BodySnifferThrottle allows intercepting and modifying response content. Need to be careful that this cannot be abused by pages to bypass security restrictions. The specific BodyHandler implementations should validate URLs/content.