kibana icon indicating copy to clipboard operation
kibana copied to clipboard
verified publisher icon elastic

Update vega related modules (main)

Open elastic-renovate-prod[bot] opened this issue 1 year ago • 1 comments

This PR contains the following updates:

Package Type Update Change Pending
vega dependencies minor ^5.24.0 -> ^5.30.0
vega-lite (source) dependencies minor ^5.5.0 -> ^5.20.1 5.21.0
vega-tooltip dependencies minor ^0.28.0 -> ^0.34.0

Release Notes

vega/vega (vega)

v5.30.0

Compare Source

Changes since v5.29.0

vega-functions

docs

monorepo

v5.29.0

Compare Source

docs

monorepo

vega-encode

vega-scale

vega-scenegraph

vega-typings

vega-view

  • turn off all handlers in View.finalize() to fix memory leak. (via #​3896) (Thanks @​cmerrick!)

v5.28.0

Compare Source

changes from v5.27.0

docs

vega-parser

vega-scenegraph

  • convert some of the scenegraph types to classes (via #​3864). (Thanks @​lsh!)

vega-typings

v5.27.0

Compare Source

changes from v5.26.1:

docs

monorepo

vega-scenegraph

  • Fix blurry zoom from caching devicePixelRatio (via #​3844). (Thanks @​lsh!)

vega-transforms

vega-typings

vega-view

  • Add option to resize on devicePixelRatio change (via #​3844). (Thanks @​lsh!)

v5.26.1

Compare Source

Changes from v5.26.1:

vega-scenegraph

  • Fix CanvasHandler to emit mouse over/move/out events. (#​3825)

vega-typings

  • Expose aggregate parameters in typings. (thanks @​Xitian9)

v5.26.0

Compare Source

Changes from v5.25.0:

vega-functions

  • Add geoScale expression function. (thanks @​binste)

vega-scale

  • Fix tickMinStep calculation. (thanks @​kanitw)

vega-scenegraph

  • Add experimental hybrid canvas/SVG renderer (thanks @​jonmmease)
  • Fall back to textMetrics.width for user defined width function (thanks @​jonmmease)

vega-selections

vega-transforms

  • Add aggregate parameters to vega-transform, and exponential moving average. (thanks @​Xitian9)

vega-typings

vega-voronoi

v5.25.0

Compare Source

Changes from v5.24.0:

monorepo

  • Fix browser lists for IE support. (thanks @​domoritz!)
  • Update dependencies.

vega-cli

vega-expression

vega-functions

  • Update to use hypot.

vega-regression

  • Allow zero-valued order for polynomial regression.

vega-statistics

  • Add constant regression method.

vega-transforms

  • Use robust (null proto) object for aggregate cell map. (#​3695)

vega-util

vega/vega-lite (vega-lite)

v5.20.1

Compare Source

Bug Fixes

v5.20.0

Compare Source

Bug Fixes
  • for issue when there are multiple aggregates with the same data field and aggregation operation (#​9320) (da84469), closes #​6184
  • make tick size responsive to plot height (#​9395) (3f5dce6)
Features

v5.19.0

Compare Source

Bug Fixes
Features
  • more options and examples for how marks and scales represent invalid data (e.g., nulls, NaNs) (#​9342) (502db1d), closes #​8254

5.18.1 (2024-05-07)

Bug Fixes

v5.18.1

Compare Source

Bug Fixes

v5.18.0

Compare Source

Bug Fixes
Features
  • add explicit option to control how densities are resolved, change how densities are resolved by default (#​9172) (bf0b8d3)

v5.17.0

Compare Source

Bug Fixes
Features

5.16.3 (2023-11-10)

Bug Fixes

5.16.2 (2023-11-08)

Bug Fixes

5.16.1 (2023-10-12)

Bug Fixes

v5.16.3

Compare Source

Bug Fixes

v5.16.2

Compare Source

Bug Fixes

v5.16.1

Compare Source

Bug Fixes

v5.16.0

Compare Source

Bug Fixes
Features
  • make bar & rect marks support bandPosition (allow center-alignment with ticks) (#​9126) (bd014eb)
  • swap 'mouse' events to 'pointer' events for mobile compatibility; update site and docs (#​9089) (fa97659)

5.15.1 (2023-10-02)

Bug Fixes
  • correct the formula for interpolating between bin start and end (interpolatedSignalRef) (#​9111) (d4b27bb)
  • use resolve "shared" instead of steps to fix artifacts in grouped density transform (#​9106) (06e63e6)

v5.15.1

Compare Source

Bug Fixes
  • correct the formula for interpolating between bin start and end (interpolatedSignalRef) (#​9111) (d4b27bb)
  • use resolve "shared" instead of steps to fix artifacts in grouped density transform (#​9106) (06e63e6)

v5.15.0

Compare Source

Bug Fixes
  • unescape paths in binned timeUnits to allow for fields with periods in name (#​9088) (53ede0e)
  • update ts and fix typing issues (#​9066) (b421a44)
Features

5.14.1 (2023-07-20)

Bug Fixes
  • Area with plain quantitative fields on both axes get stacked by default (#​9018) (b79bc9f)

v5.14.1

Compare Source

Bug Fixes
  • Area with plain quantitative fields on both axes get stacked by default (#​9018) (b79bc9f)

v5.14.0

Compare Source

Bug Fixes
Features

v5.13.0

Compare Source

Features
  • support domainRaw so one may bind domainRaw to a parameter to build custom interaction (#​8989) (21c823d)

v5.12.0

Compare Source

Features

5.11.1 (2023-06-26)

Bug Fixes
  • show gridline of first axis only for dual axes charts (#​8962) (a522a87)

v5.11.1

Compare Source

Bug Fixes
  • show gridline of first axis only for dual axes charts (#​8962) (a522a87)

v5.11.0

Compare Source

Bug Fixes
Features

v5.10.0

Compare Source

Bug Fixes
Features

5.9.3 (2023-05-18)

Bug Fixes

5.9.2 (2023-05-17)

Bug Fixes
  • make durationExpr work for week (#​8907) (bf99571)
  • respect bandWithNestedOffsetPaddingInner for grouped bar within discretized temporal axis (#​8906) (9050e6b)

5.9.1 (2023-05-15)

Bug Fixes

v5.9.3

Compare Source

Bug Fixes

v5.9.2

Compare Source

Bug Fixes
  • make durationExpr work for week (#​8907) (bf99571)
  • respect bandWithNestedOffsetPaddingInner for grouped bar within discretized temporal axis (#​8906) (9050e6b)

v5.9.1

Compare Source

Bug Fixes

v5.9.0

Compare Source

Features

v5.8.0

Compare Source

Bug Fixes
  • avoid repetitive labels by using tickMinStep (#​8872) (0b1c385)
  • remove incorrect stack transform from charts with one linear Q-axis one non-linear Q-axis. (#​8871) (f4d928a)
Features

5.7.1 (2023-04-17)

v5.7.1

Compare Source

v5.7.0

Compare Source

Bug Fixes
  • add repeat prefix to name in normalization (#​8733) (2ea3265)
  • allow setting width/height in repeated layers (#​8723) (9e26410)
  • correctly apply stacked to bar with quantitative x and quantitative y axes (#​8838) (4b9d22c)
  • correctly handle merging of domains with more than one sort object (#​8567) (1eedb8f)
  • move name from layer chart to one of the inside layers (#​8662) (d99f614)
  • orient logic for when bar with x=T + simplify logic (#​8739) (ea4aa64)
  • prevent duplicate spec names in top-level selection view paths (#​8486) (ba46bd2)
Features
  • enable interval selections for cartographic projections (#​6953) (e5cbe8f)

5.6.1 (2023-02-13)

5.6.0 (2022-10-14)

  • Bump version to: 5.6.0 [skip ci] (1ff500a)
  • Update CHANGELOG.md [skip ci] (71a7e87)

5.5.0 (2022-08-15)

  • Bump version to: 5.5.0 [skip ci] (0a2b57f)
  • Update CHANGELOG.md [skip ci] (6562bea)

v5.6.1

Compare Source

v5.6.0

Compare Source

:tada: This release contains work from new contributors! :tada:

Thanks for all your work!

:heart: Lam Thien Toan (@​danielmalaton)

:heart: Brandon Hamilton (@​mistidoi)

:heart: null@​sl-solution

:heart: Zachary Bys (@​ZacharyBys)

🚀 Enhancement
🐛 Bug Fix

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

  • [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

elastic-renovate-prod[bot] avatar Aug 30 '24 16:08 elastic-renovate-prod[bot]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

elasticmachine avatar Aug 30 '24 16:08 elasticmachine

I'm still having transpiling problems here due to vega-lite and vega-tooltip when testing it...

ERROR in ./node_modules/vega-lite/build/vega-lite.js 330:26
       │          Module parse failed: Unexpected token (330:26)
       │          You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
       │          |     }
       │          |     for (const s of src) {
       │          >       deepMerge_(dest, s ?? {});
       │          |     }
       │          |     return dest;
       │           @ ./public/data_model/vega_parser.ts 18:0-64 191:25-32 217:16-23 457:38-53
       │           @ ./public/vega_request_handler.ts
       │           @ ./public/async_services.ts
       │           @ ./public/vega_fn.ts
       │           @ ./public/plugin.ts
       │           @ ./public/index.ts

ERROR in ./node_modules/vega-tooltip/build/vega-tooltip.js 389:61
       │          Module parse failed: Unexpected token (389:61)
       │          You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
       │          |         this.el.setAttribute('id', this.options.id);
       │          |         this.el.classList.add('vg-tooltip');
       │          >         const tooltipContainer = document.fullscreenElement ?? document.body;
       │          |         tooltipContainer.appendChild(this.el);
       │          |       }
       │           @ ./public/vega_view/vega_tooltip.js 10:0-67 66:19-39
       │           @ ./public/vega_view/vega_base_view.js
       │           @ ./public/vega_view/vega_view.js
       │           @ ./public/vega_visualization.ts
       │           @ ./public/components/vega_vis_component.tsx
       │           @ ./public/async_services.ts
       │           @ ./public/vega_fn.ts
       │           @ ./public/plugin.ts
       │           @ ./public/index.ts

@elastic/kibana-core can we provide some additional flags for these 2 packages to solve this?

dej611 avatar Sep 05 '24 13:09 dej611

Unfortunately, vega-lite and vega-tooltip no-longer provide an es5-compatible build, and we have been attempting to avoid parsing/transpiling 3rd parties (with some exceptions).

Applying this patch would use our babel transpiler stack on the problematic 2 modules, and also add a new plugin for transpiling some more new syntax ( ??= ). Don't forget to run a yarn kbn bootstrap before pushing it, because the yarn-lock might change.

diff --git a/package.json b/package.json
index ea48b0b4cf7..6db0d0c0838 100644
--- a/package.json
+++ b/package.json
@@ -1279,6 +1279,7 @@
     "@babel/plugin-proposal-optional-chaining": "^7.21.0",
     "@babel/plugin-proposal-private-methods": "^7.18.6",
     "@babel/plugin-transform-class-properties": "^7.24.7",
+    "@babel/plugin-transform-logical-assignment-operators": "^7.24.7",
     "@babel/plugin-transform-numeric-separator": "^7.24.7",
     "@babel/plugin-transform-runtime": "^7.24.7",
     "@babel/preset-env": "^7.24.7",
diff --git a/packages/kbn-babel-preset/common_preset.js b/packages/kbn-babel-preset/common_preset.js
index b1fbacc9b05..276c13460b0 100644
--- a/packages/kbn-babel-preset/common_preset.js
+++ b/packages/kbn-babel-preset/common_preset.js
@@ -39,6 +39,9 @@ module.exports = () => ({
         // Need this since we are using TypeScript 3.9+
         require.resolve('@babel/plugin-proposal-private-methods'),
 
+        // Needed for ??= usage in our vega-* dependencies
+        require.resolve('@babel/plugin-transform-logical-assignment-operators'),
+
         // It enables the @babel/runtime so we can decrease the bundle sizes of the produced outputs
         [
           require.resolve('@babel/plugin-transform-runtime'),
diff --git a/packages/kbn-optimizer/src/worker/webpack.config.ts b/packages/kbn-optimizer/src/worker/webpack.config.ts
index f81d0e89fb9..97487c07a70 100644
--- a/packages/kbn-optimizer/src/worker/webpack.config.ts
+++ b/packages/kbn-optimizer/src/worker/webpack.config.ts
@@ -258,6 +258,18 @@ export function getWebpackConfig(
             },
           },
         },
+        {
+          test: /node_modules\/vega.*\/.*\.(js|tsx?)$/,
+          exclude: /node_modules\/vega.*\/.*build-es5.*\.(js|tsx?)$/,
+          use: {
+            loader: 'babel-loader',
+            options: {
+              babelrc: false,
+              envName: worker.dist ? 'production' : 'development',
+              presets: [BABEL_PRESET],
+            },
+          },
+        },
         {
           test: /\.(html|md|txt|tmpl)$/,
           use: {

@elastic/kibana-operations - is it okay to set up a 3rd party module transpilation exception like this? I'd assume this wouldn't be the first or last module that would need transpilation - do we have a more maintainable way of listing these?

delanni avatar Sep 10 '24 12:09 delanni

@delanni I implemented your changes and the build runs and starts fine but when loading a vega visualization it throws a runtime error because the vega module that is required from node_modules/vega-lite/build/vega-lite.js is undefined. Is there something missing to correctly require modules in these 3rd party files?

@elastic/kibana-operations Are you okay with adding these changes? Otherwise we will be blocked from updating vega dependencies for a while.

nickofthyme avatar Oct 01 '24 23:10 nickofthyme

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

⚠️ Warning: custom changes will be lost.

elastic-renovate-prod[bot] avatar Oct 02 '24 08:10 elastic-renovate-prod[bot]

@delanni I implemented your changes and the build runs and starts fine but when loading a vega visualization it throws a runtime error because the vega module that is required from node_modules/vega-lite/build/vega-lite.js is undefined. Is there something missing to correctly require modules in these 3rd party files?

@elastic/kibana-operations Are you okay with adding these changes? Otherwise we will be blocked from updating vega dependencies for a while.

Works for me, mind if we get a CI run in and then I'll do a full review?

jbudz avatar Oct 02 '24 14:10 jbudz

:broken_heart: Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #56 / dashboard sample data dashboard toggle from Discover to Dashboard attempt 2
  • [job] [logs] FTR Configs #56 / dashboard sample data dashboard toggle from Discover to Dashboard attempt 2
  • [job] [logs] FTR Configs #72 / Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges
  • [job] [logs] FTR Configs #72 / Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges
  • [job] [logs] FTR Configs #35 / discover/context_awareness extension getAdditionalCellActions data view mode should not render incompatible cell action for message column
  • [job] [logs] FTR Configs #96 / home app sample data accessing sample dashboards should render visualizations
  • [job] [logs] FTR Configs #96 / home app sample data accessing sample dashboards should render visualizations
  • [job] [logs] FTR Configs #73 / Journey[web_logs_dashboard_dataview] Go to Web Logs Dashboard
  • [job] [logs] FTR Configs #73 / Journey[web_logs_dashboard_dataview] Go to Web Logs Dashboard
  • [job] [logs] FTR Configs #34 / Journey[web_logs_dashboard_esql] Go to Web Logs Dashboard
  • [job] [logs] FTR Configs #34 / Journey[web_logs_dashboard_esql] Go to Web Logs Dashboard
  • [job] [logs] FTR Configs #35 / Journey[web_logs_dashboard] Go to Web Logs Dashboard
  • [job] [logs] FTR Configs #35 / Journey[web_logs_dashboard] Go to Web Logs Dashboard
  • [job] [logs] x-pack/test_serverless/functional/test_suites/observability/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] x-pack/test_serverless/functional/test_suites/search/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] x-pack/test_serverless/functional/test_suites/security/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] x-pack/test_serverless/functional/test_suites/security/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] x-pack/test_serverless/functional/test_suites/observability/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] x-pack/test_serverless/functional/test_suites/search/common_configs/config.group2.ts / lens serverless - group 1 vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] FTR Configs #87 / machine learning - data frame analytics outlier detection creation iowa house prices navigates through the wizard and sets all needed fields
  • [job] [logs] FTR Configs #87 / machine learning - data frame analytics outlier detection creation iowa house prices navigates through the wizard and sets all needed fields
  • [job] [logs] FTR Configs #18 / visualize app vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] FTR Configs #18 / visualize app vega chart in visualize app vega chart initial render should have view and control containers
  • [job] [logs] FTR Configs #45 / Visualize smoke telemetry tests should trigger render event for vega visualization
  • [job] [logs] FTR Configs #45 / Visualize smoke telemetry tests should trigger render event for vega visualization

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
visTypeVega 272 273 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ml 4.6MB 4.6MB +45.5KB
visTypeVega 1.8MB 1.9MB +45.6KB
total +91.1KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-npmDll 6.2MB 6.2MB +173.0B

History

  • :broken_heart: Build #238892 failed c593180a5d86eacfd7669fe3da402e6a6fddce0f
  • :broken_heart: Build #238826 failed ba12ae5a669f462f2d1c915f6e6bc6fbf21bc694

To update your PR or re-run it, just comment with: @elasticmachine merge upstream

kibana-ci avatar Oct 02 '24 19:10 kibana-ci

Had another look at this and couldn't make it work. While the solution above seems to resolve the nullish coalesce syntax problem, the webpack configuration doesn't seem to correctly resolve the vega dependency in the vega-lite module, leading to this error when loading a Vega visualization:

Screenshot 2024-11-28 at 16 38 32

In Firefox the problem is actually clearer:

Screenshot 2024-11-28 at 16 39 44

The main issue arise from the UMD wrapper within vega-lite which asks for require('vega') who resolves into undefined and passed to the vega-lite code:

Screenshot 2024-11-28 at 16 52 45

@delanni any idea on how that is happening?

dej611 avatar Nov 28 '24 15:11 dej611

Unfortunately clueless. My suggested changes are supposed to communicate to the transpiler: please treat /vega.*/ with babel-loader. I don't know why the import suddenly starts to point to an empty module.

If you have time, could you try checking, if the applied loader-change would cause the same error on the present (in main) version of vega*? If the issue is present, we can assume it's related to the loader change. If it doesn't fail on main, then the error comes from the import setup in the new versions, and we have to investigate further in there.

delanni avatar Nov 29 '24 10:11 delanni

:robot: Jobs for this PR can be triggered through checkboxes. :construction:

:information_source: To trigger the CI, please tick the checkbox below :point_down:

  • [ ] Click to trigger kibana-pull-request for this PR!
  • [ ] Click to trigger kibana-deploy-project-from-pr for this PR!

elasticmachine avatar Dec 05 '24 08:12 elasticmachine

Maybe we could give it another try now that webpack 5 merged?

dej611 avatar Feb 20 '25 11:02 dej611

Worth a try, but I guess the real difference will come when we can require ES Modules, with Node 22 (https://github.com/elastic/kibana/pull/205983)

delanni avatar Feb 20 '25 15:02 delanni

Managed to upgrade the packages with webpack 5 with no issue: https://github.com/elastic/kibana/pull/220909

dej611 avatar May 19 '25 13:05 dej611