chromiumoxide
chromiumoxide copied to clipboard
Published
20 hours ago •
mattsse
mattsse
Is there an example of Network Request Interception?
Could you provide a brief example of how to do Network Request Interception in order to extract the url of a specific xhr request?
There is one in the examples directory in this repo, called interception.rs, though I'm still struggling to understand how it works
I managed to get it working and also discovered you don't actually need to have Page in an Arc, which is usefull if you need to call a method that consumes Page, such as Page::close.
Below is the example I linked to with a couple of extra comments
use std::sync::Arc;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use chromiumoxide::cdp::browser_protocol::fetch::{
ContinueRequestParams, EventRequestPaused, FulfillRequestParams,
};
// Required to call `next` on `handler` and `request_paused`
use futures::StreamExt;
use chromiumoxide::browser::{Browser, BrowserConfig};
const CONTENT: &str = "<html><head></head><body><h1>TEST</h1></body></html>";
const TARGET: &str = "https://news.ycombinator.com/";
#[async_std::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// The `tracing_subscriber` crate is required. Without it, the requests won't actually
// be paused. Make sure to only invoke this once in your program or it will panic,
// having it on `main` helps avoiding that
tracing_subscriber::fmt::init();
// Spawn browser
let (mut browser, mut handler) = Browser::launch(
BrowserConfig::builder()
.enable_request_intercept()
.disable_cache()
.build()?,
)
.await?;
let browser_handle = async_std::task::spawn(async move {
while let Some(h) = handler.next().await {
if h.is_err() {
break;
}
}
});
// Setup request interception
// `Arc` is optional, but I didnt want to remove it because I'm not compiling the code
let page = Arc::new(browser.new_page("about:blank").await?);
let mut request_paused = page.event_listener::<EventRequestPaused>().await.unwrap();
let intercept_page = page.clone();
let intercept_handle = async_std::task::spawn(async move {
while let Some(event) = request_paused.next().await {
// You MUST do `intercept_page.execute` with either
// `FulfillRequestParams`, `ContinueRequestParams` or `FailRequestParams`
// for ALL requests. Any request that isn't treated will be permanently stuck
// here, which will crash your program
if event.request.url == TARGET {
if let Err(e) = intercept_page
.execute(
FulfillRequestParams::builder()
.request_id(event.request_id.clone())
.body(BASE64_STANDARD.encode(CONTENT))
.response_code(200)
.build()
.unwrap(), // Will panic if `request_id`, `body` or `response_code` are missing. Other fields are optional
)
.await
{
println!("Failed to fullfill request: {e}");
}
} else if let Err(e) = intercept_page
.execute(ContinueRequestParams::new(event.request_id.clone()))
.await
{
println!("Failed to continue request: {e}");
}
}
});
// Navigate to target
page.goto(TARGET).await?;
page.wait_for_navigation().await?;
let content = page.content().await?;
if content == CONTENT {
println!("Content overriden!")
}
// Navigate to other
page.goto("https://google.com").await?;
page.wait_for_navigation().await?;
let content = page.content().await?;
if content != CONTENT {
println!("Content not overriden!")
}
browser.close().await?;
browser_handle.await;
intercept_handle.await;
Ok(())
}