Initializing... RangeError: invalid array length

[ioweb-gr]

Hi,

I created a full report via CLI and then tried to view it in the control panel UI however it throws this error

Initializing... RangeError: invalid array length

What does it mean?

[NoiseByNorthwest]

Hi, This the bare exception message which is displayed here. Is there some related logs in the javascript console ?

[ioweb-gr]

Unfortunately there's nothing in the console log. That's why I just sent this.

[NoiseByNorthwest]

I've just added a loading errors log on master. Can you try with it and show me the error details in the console?

[ioweb-gr]

Sure here's the output from console.log

RangeError: "invalid array length"
    PackedRecordArray https://dev.example.com/?SPX_UI_URI=/js/utils.js:166
    CallList https://dev.example.com/?SPX_UI_URI=/js/profileData.js:334
    setMetadata https://dev.example.com/?SPX_UI_URI=/js/profileData.js:1242
    <anonymous> https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.gr-16487-1179048481:143
    promise callback* https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.gr-16487-1179048481:142
    jQuery 13
        j
        k
        setTimeout handler*g/<
        i
        fireWith
        fire
        i
        fireWith
        ready
        S
        EventListener.handleEvent*
        <anonymous>
        <anonymous>
phpinfo.php:296:29
    <anonymous> https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.gr-16487-1179048481:296
    (Async: promise callback)
    <anonymous> https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.gr-16487-1179048481:294
    jQuery 13
        j
        k
        (Async: setTimeout handler)
    g
        i
        fireWith
        fire
        i
        fireWith
        ready
        S
        (Async: EventListener.handleEvent)
    <anonymous>
        <anonymous>
        <anonymous>

[NoiseByNorthwest]

Thanks, can you try again with the tmp_ui_logs branch which adds another temporary log ? And what is the size of the report in terms of recorded calls ?

[ioweb-gr]

Sure here it is

ArrayBuffer size 4836452376 86365221 56 dev..gr:166:17
RangeError: "invalid array length"
    PackedRecordArray https://dev.example.com/?SPX_UI_URI=/js/utils.js:167
    CallList https://dev.example.com/?SPX_UI_URI=/js/profileData.js:334
    setMetadata https://dev.example.com/?SPX_UI_URI=/js/profileData.js:1242
    <anonymous> https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.com-16487-1179048481:143
    promise callback* https://dev.example.com/phpinfo.php?SPX_UI_URI=/report.html&key=spx-full-20200331_221734-vhosting02.example.com-16487-1179048481:142
    jQuery 13

The other information I'm assuming is in the json file so here it is

{
  "key": "spx-full-20200331_221734-vhosting02.example.gr-16487-1179048481",
  "exec_ts": 1585693054,
  "host_name": "vhosting02.example.gr",
  "process_pid": 16487,
  "process_tid": 16487,
  "process_pwd": "/httpdocs",
  "cli": 1,
  "cli_command_line": "/usr/local/bin/n98-magerun sys:cron:run evirtual_product_xautoimportall",
  "http_request_uri": "n/a",
  "http_method": "n/a",
  "http_host": "n/a",
  "wall_time_ms": 1462586425,
  "peak_memory_usage": 73039448,
  "called_function_count": 1896,
  "call_count": 86365221,
  "recorded_call_count": 86365221,
  "enabled_metrics": [
    "wt"
    ,"zm"
  ]
}

[NoiseByNorthwest]

86M of function calls is quite huge, the web UI is designed to scale to fiew million of calls and has a hard limit of 76M calls. I've not planned to increase the report size capacity on UI side as the current limit seems to be enough oversized for the most common use cases.

Did you consider enabling sampling mode in order to contain the report size?

[ioweb-gr]

I didn't know there was a hard limit on this. I'll try changing the sampling mode and see if it works. Are you sure the error is coming due to this?

[NoiseByNorthwest]

Yes, you can also try to replace the "float64" occurences by "float32" as a space / accuracy trade-off in this loop https://github.com/NoiseByNorthwest/php-spx/blob/master/assets/web-ui/js/profileData.js#L328 and then load your big report again

This hard limit is not documented, simply because I thought that the practical limit was far below.

[ioweb-gr]

I'll give it a shot and let you know by using the workaround first. Would it be possible to add an error log in the console if the hard limit is exceeded?

[NoiseByNorthwest]

In the javascript console or in place of the cryptic "RangeError..." in the progress view, yes.

[NoiseByNorthwest]

The actual hard limit is in fact 38M of function calls, and 67M of function calls with the workaround, thus still not enough for your report's size.

[ioweb-gr]

I'll try it when I get on my PC. I think this type of message should show up in the interface directly as it's not really an error

[NoiseByNorthwest]

The limit can also be almost double up if the memory usage is not analyzed, leaving only the wall time as the analyzed metric of your report. We can go further by allowing to skip short calls for big reports like yours. It could be configurable via a pre-analyze form for big reports only, where we could choose the analyzed metrics and the threshold for short calls eviction.

[ioweb-gr]

It actually hit the same limitation after the change as you said. I think a threshold option to remove short calls would be very useful option. I've seen it in many profiler apps before.

This could also be used as a filter in the UI to more easily locate what affects the performance.