Taking a screenshot of the full page

[viebel]

Is there a way to take a screenshot of the full page and not only of the view port?

[lread]

Good question, I'll poke around and get back to you.

[lread]

The W3C WebDriver spec talks about screen captures for visible content and the viewport.

But there are often workarounds, and sometimes webdriver implementation extras outside of the w3c spec.

For example, I just learned that the firefox webdriver supports a custom feature to take full page screenshots.

We can use Etaoin's execute to invoke the Firefox-specific full page screen capture:

(require '[etaoin.api :as e]
         '[clojure.java.io :as io])

(import '(java.util Base64))

(e/with-firefox driver
  (e/go driver "https://www.w3.org/TR/webdriver")
  (let [resp (e/execute {:driver driver
                         :method :get
                         :path [:session (:session driver) "moz" "screenshot" "full"]})
        b64str (some-> resp :value)]
    (with-open [out (io/output-stream "full-shot.png")]
      (.write out ^bytes
              (-> (Base64/getDecoder)
                  (.decode b64str))))))

Does that do the trick for you? Or do you need a chrome-based solution?

[viebel]

Thank yo @lread!

I didn't know that only screen capture for visible content was mentioned in the W3C spec.

As you have guessed I need a chrome-based solution.

[lread]

Hiya @viebel, I did a little poking around and it seems that chromedriver also has the extra support for full-page screenshots.

I tested via:

(require '[etaoin.api :as e]
         '[clojure.java.io :as io])

(import '(java.util Base64))

(e/with-chrome-headless driver
  (e/go driver "https://en.wikipedia.org/wiki/Clojure")
  (let [resp (e/execute {:driver driver
                         :method :get
                         :path [:session (:session driver) "screenshot" "full"]})
        b64str (some-> resp :value)]
    (with-open [out (io/output-stream "full-shot-chrome.png")]
      (.write out ^bytes
              (-> (Base64/getDecoder)
                  (.decode b64str))))))

[lread]

The edge driver is based on the chrome driver and has this support too. I suppose we could add explicit support for full-page screenshots into Etaoin.

Safari would be the unsupported outlier (which is not terribly unusual for safari).

[viebel]

Thank you so much for providing a code snippet for Chrome.

Is it on purpose that the :path for Firefox contains "moz" between (:session driver) and "screenshot" while for Chrome, there is nothing between (:session driver) and "screenshot"?

[lread]

Is it on purpose that the :path for Firefox contains "moz" between (:session driver) and "screenshot" while for Chrome, there is nothing between (:session driver) and "screenshot"?

Yes, I guess the Firefox and Chrome teams both went their own ways in URL design for a custom feature.

Please let me know if you find any major issues with full-page screenshots. If it works well enough, we can consider adding it to the Etaoin API.

[viebel]

I am not able to take a full page screenshot with Chrome, only with Chrome headless.

Here's what I tried:

(def driver (e/chrome))

(let [resp (e/execute {:driver driver
                       :method :get
                       :path [:session (:session driver) "screenshot" "full"]})
      b64str (some-> resp :value)]
  (with-open [out (io/output-stream "full-shot-chrome.png")]
    (.write out ^bytes
            (-> (Base64/getDecoder)
                (.decode b64str)))))

And I got the following error:

; (err) Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
; (err) throw+: {:response {:sessionId "23192dc589a6c542b98e5bf5419acfd8", :status 13, :value {:message "unknown error: unhandled inspector error: {\"code\":-32000,\"message\":\"Unable to capture screenshot\"}\n  (Session info: chrome=103.0.5060.134)\n  (Driver info: chromedriver=103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}),platform=Mac OS X 12.5.0 x86_64)"}}, :path "session/23192dc589a6c542b98e5bf5419acfd8/screenshot/full", :payload nil, :method :get, :type :etaoin/http-error, :port 65031, :host "127.0.0.1", :status 200, :driver {:args ("chromedriver" "--port=65031"), :capabilities {:loggingPrefs {:browser "ALL"}}, :process #object[java.lang.ProcessImpl 0x27a41cd6 "Process[pid=82046, exitValue=\"not exited\"]"], :locator "xpath", :type :chrome, :env nil, :port 65031, :host "127.0.0.1", :url "http://127.0.0.1:65031", :session "23192dc589a6c542b98e5bf5419acfd8"}}

When I replace (e/chrome) with (e/chrome-headless), it works fine:

(def driver (e/chrome-headless))

(let [resp (e/execute {:driver driver
                       :method :get
                       :path [:session (:session driver) "screenshot" "full"]})
      b64str (some-> resp :value)]
  (with-open [out (io/output-stream "full-shot-chrome.png")]
    (.write out ^bytes
            (-> (Base64/getDecoder)
                (.decode b64str)))))

[lread]

Thanks for sharing your findings, @viebel!

Your e/chrome code example works for me, I'm running:

• Etaoin master
• macOS 10.15.7
• chromedriver 103.0.5060.134
• Chrome 103.0.5060.134
• Temurin JDK 17

One small thing I noticed about your example is that you are capturing a blank page; you have not used e/go to navigate anywhere. This worked when I tried it and captured a white blank page for me, but thought it worth mentioning.

[viebel]

One small thing I noticed about your example is that you are capturing a blank page; you have not used e/go to navigate anywhere. This worked when I tried it and captured a white blank page for me, but thought it worth mentioning.

I made a mistake when I copy pasted my code snippet.

Your e/chrome code example works for me, I'm running:

With etatoin master (cd7bd418c0c6e6947456e52140a46e60d9bfcbdd), it works. But not with the version 0.4.6 from Clojars.

[lread]

That seems a bit odd (or maybe not, I've made many changes since 0.4.6), I'll cut a new release sometime soon.

[lread]

That seems a bit odd (or maybe not, I've made many changes since 0.4.6), I'll cut a new release sometime soon.

Note: I did cut that release back in August.

[viebel]

Thank you!

[lread]

@viebel, I didn't add this feature! 🙂

I was only responding to your aside:

With etatoin master (cd7bd418c0c6e6947456e52140a46e60d9bfcbdd), it works. But not with the version 0.4.6 from Clojars.

Shall we re-open?

[viebel]

My bad. Re-opened.

[lread]

I suppose we should typically be matching features of the w3c spec. This way we are less likely to suffer from breakages to changes in vendor-specific features. A full page screenshot is not currently part of the w3c spec. So, I think I'll close this one.