Try using Kaleido v1 for image tests

[emilykl]

[ayjayt]

Hi @emilykl, any interest in assigning me as well?

[emilykl]

@ayjayt You're welcome to try on your own branch if you like -- I believe this task is blocked by https://github.com/plotly/plotly.py/pull/5207 though which I'm still testing

[emilykl]

@ayjayt There are two figures which are failing consistently with an error in Kaleido v1 -- could you investigate?

These are the two figures:

gl3d_streamtube_reversed_ranges
mathjax

(There are a few others which are failing intermittently but I think that's due to instability in the pipeline, not anything to do with Kaleido)

Here is an archive containing:

1. The JSON figure representations for the failing figures
2. A script which attempts to generate an image from each, and prints the traceback on failure.

failing_figures_kaleido_v1.zip

You'll need to pip install a few things in your environment to run the script:

python -m pip install "git+https://github.com/plotly/plotly.py.git@6837831"
python -m pip install "git+https://github.com/plotly/[email protected]#subdirectory=src/py"
python -m pip install numpy

Below is the output generated by running python generate_images.py from inside the root directory of the archive:

Error output

(make_baseline) (base) ekl@Emilys-MacBook-Air-2 failing_figures_kaleido_v1 % python generate_images.py
=============
=== Failed to generate image for gl3d_streamtube_reversed_ranges ===
=============
Traceback (most recent call last):
  File "generate_images.py", line 32, in <module>
    pio.write_image(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/plotly/io/_kaleido.py", line 509, in write_image
    img_data = to_image(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/plotly/io/_kaleido.py", line 379, in to_image
    img_bytes = kaleido.calc_fig_sync(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 145, in calc_fig_sync
    return _async_thread_run(calc_fig, args=args, kwargs=kwargs)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 138, in _async_thread_run
    raise res
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 129, in run
    q.put(asyncio.run(func(*args, **kwargs)))
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 55, in calc_fig
    return await k.calc_fig(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/kaleido.py", line 319, in calc_fig
    spec, full_path = build_fig_spec(fig, path, opts)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/_fig_tools.py", line 144, in build_fig_spec
    name = _next_filename(directory, prefix, ext)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/_fig_tools.py", line 94, in _next_filename
    re_number = re.compile(r"^" + prefix + r"-(\d+)\." + ext + r"$")
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/re.py", line 252, in compile
    return _compile(pattern, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 598, in _parse
    raise source.error(msg, len(this) + 1 + len(that))
re.error: bad character range _-2 at position 94
=============
=============
=== Failed to generate image for mathjax ===
=============
Traceback (most recent call last):
  File "generate_images.py", line 32, in <module>
    pio.write_image(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/plotly/io/_kaleido.py", line 509, in write_image
    img_data = to_image(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/plotly/io/_kaleido.py", line 379, in to_image
    img_bytes = kaleido.calc_fig_sync(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 145, in calc_fig_sync
    return _async_thread_run(calc_fig, args=args, kwargs=kwargs)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 138, in _async_thread_run
    raise res
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 129, in run
    q.put(asyncio.run(func(*args, **kwargs)))
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/__init__.py", line 55, in calc_fig
    return await k.calc_fig(
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/kaleido.py", line 319, in calc_fig
    spec, full_path = build_fig_spec(fig, path, opts)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/_fig_tools.py", line 144, in build_fig_spec
    name = _next_filename(directory, prefix, ext)
  File "/Users/ekl/code/plotly.js/venv/make_baseline/lib/python3.8/site-packages/kaleido/_fig_tools.py", line 94, in _next_filename
    re_number = re.compile(r"^" + prefix + r"-(\d+)\." + ext + r"$")
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/re.py", line 252, in compile
    return _compile(pattern, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 525, in _parse
    code = _escape(source, this, state)
  File "/Users/ekl/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/lib/python3.8/sre_parse.py", line 426, in _escape
    raise source.error("bad escape %s" % escape, len(escape))
re.error: bad escape \h at position 3
=============

[ayjayt]

@emilykl

https://github.com/plotly/Kaleido/pull/339 makes it work for me

here's a uv script preamble to build in the build steps (will redownload stuff) right in generate_images

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "kaleido",
#     "numpy",
#     "plotly",
# ]
#
# [tool.uv.sources]
# plotly = { git = "https://github.com/plotly/plotly.py.git", rev = "6837831" }
# ///

[archmoj]

@emilykl After I viewed most significant changes on my dev branch I suggest you drop the a426afa commit (possibly by force pushing i.e. to reduce the size of changes) and instead commit the changes of the following baselines:

test/image/baselines/mathjax.png
test/image/baselines/ternary-mathjax-title-place-subtitle.png
test/image/baselines/legend_mathjax_title_and_items.png
test/image/baselines/world-cals.png

test/image/baselines/worldcup.png
test/image/baselines/funnel_horizontal_group_basic.png
test/image/baselines/font-decorline-scatter.png
test/image/baselines/text_on_shapes_basic.png
test/image/baselines/indicator_scatter.png

Thank you :pray:

[emilykl]

Closing in favor of https://github.com/plotly/plotly.js/pull/7498.