chromote icon indicating copy to clipboard operation
chromote copied to clipboard

Problem with chromote package

Open sale4cast opened this issue 8 months ago • 0 comments

Dear community,

First of all, I would like to thanks everyone for your previous support. I want to extract "hotel prices" from google. From your previous suggestions, I can extract the first page prices perfectly. However, I want to extract also second page prices. Here I attach the code below:

library(shiny)
library(curl)
library(chromote)
library(pagedown)

ui <- fluidPage(
  textOutput("targetId"),
  tableOutput("result")
)

r <- Chromote$new()

server <- function(session, input, output) {
  driver <- r$new_session()
  output$targetId <- renderText(paste0("Target ID: ", driver$get_target_id()))
  p <- driver$Page$loadEventFired(wait_ = FALSE)
  driver$Page$navigate("https://google.com", wait_ = FALSE)
  
  p$then(function(value){
    googleSearchText <- "4 star hotel in barcelona"
    driver$Runtime$evaluate(paste0('document.querySelector("textarea").value = "', googleSearchText,'"'))
    
    {
      p2 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("input[aria-label=\'Google Search\']").click()', wait_ = FALSE)
    }
    
    p2
  })$then(function(value){
    driver$Runtime$evaluate('document.querySelector("div.R2w7Jd").click()')
    {
      p3 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("div.JWXKNd").click()', wait_ = FALSE)
    }
    p3
  })$then(function(value){
    priceElement <- driver$Runtime$evaluate(
      'var elements = document.querySelectorAll(".K1smNd > c-wiz[jsrenderer=\'hAbFdb\'] .PwV1Ac");
                 var elementPrices = [];
                 elements.forEach(function(element) {
                   elementPrices.push(element.innerText);
                 });
                 elementPrices.join("@");'
    )
    driver$close()
    
    splittedPriceElements <- unlist(strsplit(priceElement$result$value, "@"))
    
    neighborHotelRoomPrices <- lapply(splittedPriceElements, function(aElement){
      ## Great Deal\n$80    Deal 5%\n$90
      roomPrice <- unlist(strsplit(aElement,"\n"))
      if(length(roomPrice) > 1){
        return(roomPrice[[2]])
      }
      return (aElement)
    })
    output$result <- renderTable(as.data.frame(unlist(neighborHotelRoomPrices)))
  })
}


shinyApp(ui, server)

Now, I want to extract second page prices as like the screenshot below:

Screenshot_368

Question: How can I extract second page prices in asynchronous way?

Note: Here I extract prices from a single page url and data updated in the particular results section only.

Best Regards, SaleForecast

sale4cast avatar Jun 23 '24 18:06 sale4cast