plotly.rs icon indicating copy to clipboard operation
plotly.rs copied to clipboard

Kaleido doesn't generate the image output on MacOS

Open alceal opened this issue 1 year ago • 11 comments

I'm using the example from the README.md to test the write_image function, but it neither generates nor panics. I'm on MacOS 15.0.1 (Sequoia).

plotly = { version = "0.10.0", features = ["kaleido"]}
use plotly::{ImageFormat, Plot, Scatter};

fn main() {
    let mut plot = Plot::new();
    let trace = Scatter::new(vec![0, 1, 2], vec![2, 1, 0]);
    plot.add_trace(trace);

    plot.write_image("out.png", ImageFormat::PNG, 800, 600, 1.0);
}

alceal avatar Nov 02 '24 14:11 alceal

@alceal thanks for reporting the issue. I was under the false impression that only the CI had issues. This means that indeed there is something wrong on MacOS. I will try to see if I can get to the bottom of it using the GitHub runners (don't own a Mac) . It might also be a Kaleido issue in itself, not a plotly-rs one, but worth looking into.

andrei-ng avatar Nov 04 '24 07:11 andrei-ng

Just a heads up, we're working on a rebuild of Kaleido. The new versions are on PyPi as v1.0.0rc0. See: https://github.com/plotly/kaleido for a bit more. I'm not sure how Kaleido is integrated into plotly.rs but if there is a coupling on the PyPi project this may have broken it.

We released a few minors (v0.4.0-v0.4.2) but we yanked them as we worked out naming scheme: This is a complete rebuild so we might end up publishing it under a new name.

The timing of this ticket is about when we published the new versions, but as they're now yanked I wouldn't expect further issues.

ndrezn avatar Nov 22 '24 21:11 ndrezn

Hi Nathan, thank you for this info. I noticed that Greg commented on a few tickets on other projects related to Kaleido that a new version is coming out and I am looking forward to that. There is something broken in Kaleido or our integration with which causes this issue on plotly-rs, just didn't have the time to look into it. Will have to take the leap soon.

We are using an older version though. Need to check if the newer release candidate improves things. FYI , in this crate, Kaleido gets downloaded at build time on the users machine and the package version is hardcoded here https://github.com/plotly/plotly.rs/blob/main/plotly_kaleido/build.rs

andrei-ng avatar Nov 23 '24 08:11 andrei-ng

Hi @alceal, is the architecture of your Mac by any chance arm?

andrei-ng avatar Nov 23 '24 15:11 andrei-ng

Hi @alceal, is the architecture of your Mac by any chance arm?

@andrei-ng Not the person you asked, but I wanted to share that I tried this on my Macbook Air to verify. It does indeed hang (no panic and nothing generated), just as described in this issue.

MacOS Sonoma 14.1.1 (23B81) Arm CPU (the M1 Apple silicon)

emilbratt avatar Nov 28 '24 00:11 emilbratt

Hi @alceal, is the architecture of your Mac by any chance arm?

@andrei-ng Not the person you asked, but I wanted to share that I tried this on my Macbook Air to verify. It does indeed hang (no panic and nothing generated), just as described in this issue.

MacOS Sonoma 14.1.1 (23B81) Arm CPU (the M1 Apple silicon)

Thank you @emilbratt!

andrei-ng avatar Nov 30 '24 09:11 andrei-ng

Been trying to track down the problem. It seems that is an Apple M1 Sillicon issue with Kaleido itslef. I asked a colleague to test it on his MacOS and he had no issues with Apple M3 Max . I am inclined to consider it to be a Kaleido issue as there have been some reports in the past related to that https://github.com/plotly/Kaleido/issues/129

I will investigate further and provide another update once I know more.

andrei-ng avatar Dec 06 '24 12:12 andrei-ng

Hi @alceal, is the architecture of your Mac by any chance arm?

Yes, it's Apple Silicon M1 Max

alceal avatar Jan 03 '25 22:01 alceal

Thank you for confirming.

andrei-ng avatar Jan 04 '25 08:01 andrei-ng

Extra info – new failing test (Apple Silicon, macOS 15.4.1 / Rust 1.86.0)

I’ve reproduced #241 on the latest Sequoia release. The minimal test below hangs and spits the familiar locale_file_path.empty() warning before Kaleido gives up.

#[cfg(target_os = "macos")]
#[cfg(feature = "kaleido")]
#[test]
fn save_surface_to_png() {
    use plotly::{ImageFormat, Plot, Surface};
    use std::path::PathBuf;

    let mut plot = Plot::new();
    let surface = Surface::new(
        vec![
            vec![1.0, 2.0, 3.0],
            vec![4.0, 5.0, 6.0],
            vec![7.0, 8.0, 9.0],
        ])
        .x(vec![1.0, 2.0, 3.0])
        .y(vec![4.0, 5.0, 6.0])
        .name("Surface");

    plot.add_trace(surface);

    let dst = PathBuf::from("out.png");
    plot.write_image(dst.to_str().unwrap(), ImageFormat::PNG, 800, 600, 1.0);

    // sanity checks
    assert!(dst.exists());
    assert!(std::fs::remove_file(&dst).is_ok());
    assert!(!dst.exists());
    assert!(!plot.to_base64(ImageFormat::PNG, 1024, 680, 1.0).is_empty()); // Fail!
}

Stderr (trimmed)

Kaleido failed to generate static image for format: png
[0511/100833.547058:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale en-US

assertion failed: !plot.to_base64(ImageFormat::PNG, 1024, 680, 1.0).is_empty()

Environment

  • macOS 15.4.1 (Sequoia) – Apple M4 Max (arm64)
  • rustc 1.86.0 (stable)
  • plotly = "0.12.1", plotly_kaleido = "0.12.1"
  • Kaleido binary Version 0.2.1

Happy to run more tests on Apple Silicon if that helps!

—Joaquín

joaquinbejar avatar May 11 '25 08:05 joaquinbejar

Created Kaleido upstream issue: https://github.com/plotly/Kaleido/issues/323

What happens

On Apple Silicon (tested on M4 Max / macOS 15.4.1), Plot::write_image fails with

{"code":525,"message":"error creating static canvas/context for image server", ...}

because plotly_kaleido starts Kaleido with this flag set:

--disable-gpu

Removing that single flag lets Kaleido 0.2.1 render the PNG just fine (see upstream issue for full logs).

Quick workaround (no code changes)

Create a tiny wrapper script called kaleido alongside it:

#!/usr/bin/env bash
DIR="$(cd "$(dirname "$0")" && pwd)"

# Strip the problematic flag
filtered=()
for arg in "$@"; do
  [[ "$arg" == "--disable-gpu" ]] && continue
  filtered+=("$arg")
done

exec "$DIR/bin/kaleido" "${filtered[@]}"

After that, write_image and to_base64 succeed and the tests pass on M‑series Macs.

Longer‑term ideas

  • Skip --disable-gpu automatically when cfg!(target_os = "macos") && cfg!(target_arch = "aarch64").
  • Or expose a PLOTLY_KALEIDO_EXTRA_ARGS env var so users can tweak flags without recompiling.

I created PR #289 to fix the issue until the new Kaleido release.

— Joaquín

joaquinbejar avatar May 11 '25 17:05 joaquinbejar

@alceal , @emilbratt , @joaquinbejar , I've implemented a different approach to perform exports using WebDriver. The latest release has now a new package plotly_static that drives a browser using WebDriver to do the static export. I've tested it in the CI pipeline for Windows, MacOS and Ubuntu. On the Windows runners it is a bit flaky, but on Linux and MacOS runners seems to be stable.

This has been introduced in PR https://github.com/plotly/plotly.rs/pull/319, would you guys be willing to test it out and give me some feedback on it?

andrei-ng avatar Jul 07 '25 21:07 andrei-ng

Hey @andrei-ng

Thanks for putting this together! I’ll try out the new plotly_static package in my Ubuntu and macOS pipelines and let you know how it goes.

Quick question: is there a specific WebDriver version you recommend or any special config to keep things running smoothly?

joaquinbejar avatar Jul 08 '25 09:07 joaquinbejar

@joaquinbejar , tricky question 😁

So far the "easiest" one that worked across these OSes was Chrome/Chromium with chromedriver. The only issue is that chromedriver needs to match the version of the installed Chrome/Chromium . Hopefully the automatic download+install of chromedriver in the plotly_static package when the feature flags is enabled works for most cases.

Firefox with geckodriver was easiest on Linux (Arch Linux, will run it on Ubuntu host soon and see if my claim holds) . Geckodriver is not so strict on version compatibility.

However, in the CI pipeline, Chrome is the better choice, as getting Firefox + geckodriver run in headless mode on CI runners has been an adventure

andrei-ng avatar Jul 08 '25 17:07 andrei-ng

@andrei-ng

I ran the example in plotly.rs/examples/static_export and it seems to work fine. I had Google Chrome already installed.
Here is a snippet of the shell output showing info about using the chromedriver.

...
warning: [email protected]: You can specify WEBDRIVER_PATH to an existing chromedriver/geckodriver installation to avoid downloads.
warning: [email protected]: You can override browser detection using BROWSER_PATH environment variable.
warning: [email protected]: chromedriver selected but not installed, will be downloaded ...
warning: [email protected]: Browser version detected: 138.0.7204.93
warning: [email protected]: Driver will be installed in: "/Users/emilbratt/.local/bin"
warning: [email protected]: Driver installed successfully ...
   Compiling plotly v0.13.1 (/Users/emilbratt/git/plotly_official/plotly.rs/plotly)
...

I am on macOS sequoia version: 15.2 (24C101). Macbook Air M1 2020.

emilbratt avatar Jul 09 '25 12:07 emilbratt

@andrei-ng

I ran the example in plotly.rs/examples/static_export and it seems to work fine. I had Google Chrome already installed. Here is a snippet of the shell output showing info about using the chromedriver.

... warning: [email protected]: You can specify WEBDRIVER_PATH to an existing chromedriver/geckodriver installation to avoid downloads. warning: [email protected]: You can override browser detection using BROWSER_PATH environment variable. warning: [email protected]: chromedriver selected but not installed, will be downloaded ... warning: [email protected]: Browser version detected: 138.0.7204.93 warning: [email protected]: Driver will be installed in: "/Users/emilbratt/.local/bin" warning: [email protected]: Driver installed successfully ... Compiling plotly v0.13.1 (/Users/emilbratt/git/plotly_official/plotly.rs/plotly) ... I am on macOS sequoia version: 15.2 (24C101). Macbook Air M1 2020.

Thank you very much for trying it out! The output lines seem sensible. Could you also check the contents of the exported images. I had this issue in my intial development phases that the PDF was empty / blank. If the exported images look fine, then we are one step closer in moving away from Kaleido

andrei-ng avatar Jul 09 '25 15:07 andrei-ng

Hey @andrei-ng, just tested the new plotly_static package on both Ubuntu 24.04 and macOS 15.5 (M4 Max). Everything works smoothly, including PDF and PNG exports—no blank files, no hangs. 👍

Really appreciate the WebDriver approach, feels much more stable than Kaleido on Apple Silicon. Let me know if you want help benchmarking or testing more edge cases!

—Joaquín

joaquinbejar avatar Jul 09 '25 17:07 joaquinbejar

@andrei-ng

Could you also check the contents of the exported images. I had this issue in my intial development phases that the PDF was empty / blank. If the exported images look fine, then we are one step closer in moving away from Kaleido

The exported images look perfectly fine including the PDF´s. :)

emilbratt avatar Jul 09 '25 18:07 emilbratt

@andrei-ng

Could you also check the contents of the exported images. I had this issue in my intial development phases that the PDF was empty / blank. If the exported images look fine, then we are one step closer in moving away from Kaleido

The exported images look perfectly fine including the PDF´s. :)

@emilbratt , thanks for confirming!

andrei-ng avatar Jul 10 '25 19:07 andrei-ng

Hey @andrei-ng, just tested the new plotly_static package on both Ubuntu 24.04 and macOS 15.5 (M4 Max). Everything works smoothly, including PDF and PNG exports—no blank files, no hangs. 👍

Really appreciate the WebDriver approach, feels much more stable than Kaleido on Apple Silicon. Let me know if you want help benchmarking or testing more edge cases!

—Joaquín

@joaquinbejar , thank you very much for testing. If you have ideas for further testing, I am happy to accept any help I can get. I am open to ideas and PRs 😉

andrei-ng avatar Jul 10 '25 19:07 andrei-ng

I am going to close this issue. I would recommend everyone tries out plotly_static sub package which uses the WebDriver approach. The old Kaleido is unmaintained and the new one is a pure python implementation that cannot be used by this crate.

andrei-ng avatar Aug 16 '25 10:08 andrei-ng