chart rendering breaks around 5000 x-Axis data points

[bvogel]

node v20 highcharts version 10.4.0 export server version: 3.1.1

Expected behaviour

Graph rendered correctly (4999 data points)

Actual behaviour

Graph rendered with increasing errors (5000 data points)

graph don't start at the same point any more

Graph broken (5001 data points)

graph is missing 3/4 of data

Reproduction steps

infiles for the cases:

working json

partially broken

broken

[bvogel]

In the meantime I found out that the error in the second graph may result from unbalanced series lengths, so that error may be expected, however the series have the same length for the third graph and it's still broken

[bvogel]

I can add an additional example for failing data around the 5000 data points mark:

working (4999 data points)

broken (5000 data points)

Although the graph is still being rendered it looses the requested gradient.

[jszuminski]

Thanks for reporting @bvogel!

It seems to be a duplicate of this issue: https://github.com/highcharts/node-export-server/issues/490

It should work in the newest version.

[bvogel]

Thank you, any news on the release date?

I'd be happy to test a beta or rc release as well

[bvogel]

Another month has passed, I can't really take this issue as resolved as we can't use the proposed solution - it still hasn't been released yet. Please advise how to proceed. (Due to internal guidelines we aren't in a position to use branches as reference for our packages)

[bvogel]

bump!

[jszuminski]

@bvogel it has been fixed (on the master branch) and on the official deployed Highcharts Export Server. Please try it out :)

[bvogel]

@jszuminski unfortunately we can't use the external server due to company policies, so I'll have to wait for the official release (my test data was deleted already)

[jszuminski]

Absolutely, I understand. If you'd like, you can always try to download the package from GitHub.

[bvogel]

Hi, I just tested with:

INFO 2024-08-08T18:05:17.669282664Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [verbose] - [cache] Fetching script - https://code.highcharts.com/11.4/highcharts-more.js
INFO 2024-08-08T18:05:17.608273297Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [verbose] - [cache] Fetching script - https://code.highcharts.com/11.4/highcharts.js
INFO 2024-08-08T18:05:17.602820134Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [cache] Updating cache version to Highcharts: 11.4/.
INFO 2024-08-08T18:05:17.602413336Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [cache] Fetching and caching Highcharts dependencies.
INFO 2024-08-08T18:05:17.600292924Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [process] Attaching exit listeners to the process.
INFO 2024-08-08T18:05:17.598449415Z [resource.labels.containerName: highcharts-export-server] Starting Highcharts Export Server v4.0.1...

but still seeing the same error. 5000 x-axis points will break rendering.

In contrast to #490 I don't get an 500 error but just a broken graph.

[bvogel]

i.e. can we have this re-opened please?

[jszuminski]

I've reopened this issue and added it to our backlog. If the problem indeed persists (we will have to investigate that), we will try to make it a part of one of the upcoming releases.

[bvogel]

Thank you!

[bvogel]

@jszuminski were you able to reproduce? should I sent test data from our end?

[jszuminski]

@bvogel if possible, you can share the test data, that would be helpful. You can also try to create a minimal reproducible example in JSFiddle, starting from there: https://jsfiddle.net/ftpyxewh/

[bvogel]

Hi @jszuminski,

this fiddle will work as fiddle but is broken if exported through the server https://jsfiddle.net/5ejsu9go/

this one will work: https://jsfiddle.net/j6ns0d3v/

Some additional data: the export server receives a post with:

json: { infile: <options from the fiddle>, scale: 2, type: "jpeg" }

[bvogel]

@jszuminski any news on this, were you able to reproduce the error we are seeing?

[jszuminski]

@bvogel I seem to have different errors when exporting this, but do not encounter what you encounter. The good news is that this https://github.com/highcharts/node-export-server/pull/561 pull request will introduce many beneficial changes and will allow to detect errors in request/CLI options if they are the ones causing an error so a part of the problem will get solved there; you will get an error or a warning message if the configuration is not optimal or not supported.

However, for now, let's get back to the main problem which is the partial rendering of the line chart. Could you please tell me:

1. Are you sending an HTTP request or are you doing this via CLI? currently, There are separate exporting processes happening; we're currently working on unifying them which will also get rid of this problem.

2. Could you share the full CLI command and a broken .json file so that I can 100% get the error that you get? I will then just run the command and share the result with you to compare. If this causes an error on my side as well, I will debug it and get to the root cause.

[bvogel]

we use a HTTP POST request, so I have no CLI or json file. The jsfiddles are very close to the requests we send. as mentioned in the previous post we use this code:

    def request_chart(for_pdf: false)
      scaling = for_pdf ? 2 : 1
      result = HTTP.post(HIGHCHARTS_EXPORT_SERVER_URL, json: { infile: @chart_options, scale: scaling, type: "jpeg" })

      raise HighchartsError, "status: #{result.status.code} reason: #{result.body}" unless result.status.success?

      StringIO.new(result.body)
    end

^^^ we see the error for for_pdf = true i.e. scale: 2

with the @chart_options being 100% identical to the ones in the fiddles. We use Highcharts Export Server v4.0.1 with Highcharts library 11.4.0

edit: this is the log from a export server for a failing request

2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Got an incoming HTTP request with ID d510f5bd30cc4ad98ad04fd4ba82ae58.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [chart] Starting the exporting process.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [chart] Attempting to export from a raw input.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Work received, starting to process.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Acquiring a worker handle.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Acquired a worker handle.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Starting work on pool entry with ID c7d29e24-f160-4ac3-befe-28c17e54e271.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Determining export path.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Treating as config.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Work completed in 129 ms.

edit 2: you can use this link to get a broken sample: https://capinside.com/products/pictet-digital-hi-eur-lu0386392772/fact_sheet?performance_start_date=2011-01-05&performance_end_date=2024-09-13&peergroup_id=13&add_personalization=false&risk_return_range=1_year and this one (one day less) for a working one https://capinside.com/products/pictet-digital-hi-eur-lu0386392772/fact_sheet?performance_start_date=2011-01-06&performance_end_date=2024-09-13&peergroup_id=13&add_personalization=false&risk_return_range=1_year - and for good measure, this is the page we start from, probably some data can be deduced from the graph in use there: https://capinside.com/p/pictet-digital-hi-eur-lu0386392772?locale=en

[jszuminski]

Okay, thank you very much @bvogel! I have successfully managed to reproduce it locally and get the same result as you do:

I will now investigate the cause of the problem by debugging what's going on inside the Puppeteer.

[bvogel]

Thank you so much for the update!

[bvogel]

@jszuminski any progress on this issue?

[bvogel]

@jszuminski mind me reminding you on this issue?