Perspective viewer with columns configuration not matching server crashes it.

[dbboileau]

Bug Report

Steps to Reproduce:

1. ... add columns property to perspective-viewer html (missing from the underlying table)
2. ... create websocket connection; perspective.websocket()
3. ... attach to perspective viewer; websocket.open_table('my_table')

Expected Result:

... missing columns ignored

Actual Result:

Tornado server crashed with segfault

signal 11 (Segmentation fault), address is 0xfffffffffffffffb from [bt]: (0) printBacktrace() [bt]: (1) sigabrt_handler(int, siginfo_t*, void*) [bt]: (2) /lib64/libpthread.so.0(+0xf5e0) [0x7f618961f5e0] [bt]: (3) /data02/apps/hfalgo_ext/376f817/gcc/gcc-9.2.0/lib64/libstdc++.so.6(_ZNSsC2ERKSs+0xb) [0x7f618119940b] [bt]: (4) perspective::t_config::t_config(perspective::t_config const&) [bt]: (5) void perspective::t_gnode::notify_contextperspective::t_ctx0(perspective::t_data_table const&, perspective::t_ctx_handle const&) [bt]: (6) /data02/apps/hfalgo_ext/376f817/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/../../perspective_python.lib/libpsp.so(+0x21a8a9) [0x7f5f7a6b98a9] [bt]: (7) /data02/apps/hfalgo_ext/376f817/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/../../perspective_python.lib/libtbb.so(+0x35b05) [0x7f5f7a47ab05] [bt]: (8) /data02/apps/hfalgo_ext/376f817/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/../../perspective_python.lib/libtbb.so(+0x35e86) [0x7f5f7a47ae86] [bt]: (9) /data02/apps/hfalgo_ext/376f817/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/../../perspective_python.lib/libtbb.so(+0x33a10) [0x7f5f7a478a10] [bt]: (10) perspective::t_gnode::notify_contexts(perspective::t_data_table const&) [bt]: (11) perspective::t_gnode::process(unsigned long) [bt]: (12) perspective::t_update_task::run() [bt]: (13) perspective::t_pool::_process_helper() [bt]: (14) perspective::t_pool::_process() [bt]: (15) /data02/apps/hfalgo_ext/376f817/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/libbinding.so(+0x6071e) [0x7f5f7bba271e]

[sc1f]

I've tried this in the streaming.py example in the Python examples folder over various pivoted and unpivoted views, and can't reproduce the segfault. Do you have a code example that will reproduce this segfault?

[dbboileau]

I encountered the issue part of a larger program (quite a lot of streaming). In order to reproduce the 'issue' made the following change to streaming.html

columns='["high", "low", "medium"]'

I got this but no segfault: WARNING:root:GC 0 views in memory

[dbboileau]

One difference I see with the example, is that we're using PerspectiveAdapter

like:

perspective_adapter = PerspectiveAdapter( port ) table = perspective_adapter.create_table( 'my_table', index='key' )

resulting stack trace:

ERROR:tornado.application:Exception in callback functools.partial(<function wrap..null_wrapper at 0x7f6012fe7510>, table_id=0) Traceback (most recent call last): │ File "/data01/home/db8109/repo/hfalgo_ext/python/Python-3.6.4/lib/python3.6/site-packages/tornado/ioloop.py", line 758, in _run_callback File "/data01/home/db8109/repo/hfalgo_ext/python/Python-3.6.4/lib/python3.6/site-packages/tornado/stack_context.py", line 300, in null_wrapper return fn(*args, **kwargs) │ File "/data01/home/db8109/repo/hfalgo_ext/python/Python-3.6.4/lib/python3.6/site-packages/perspective/table/_state.py", line 69, in call_process pool._process() MemoryError: std::bad_alloc

[timkpaine]

PerspectiveAdapter doesnt appear in our codebase, not sure we can assist. This sounds like point72 and "adapter" makes me think of reactive, maybe @robambalu has some insight into if the adapter does anything interesting

[robambalu]

Here's a version you should be able to run. i was able to repro when i have perspective running on a separate thread from the update calls. (sorry for pasting, github wont let me attach html/py )

from perspective import PerspectiveTornadoHandler, Table, PerspectiveManager

import tornado.web
import tornado.ioloop
import threading
import time

manager = PerspectiveManager()

ptable = Table( { 'foo' : int, 'key' : str  }, index='key' )
manager.host_table( 'my_table', ptable )

port =7678
s_app = tornado.web.Application([
    # create a websocket endpoint that the client Javascript can access
    (r"/websocket", PerspectiveTornadoHandler, {"manager": manager, "check_origin": True})
], websocket_ping_interval=15)
s_app.listen(port)
s_ioloop = tornado.ioloop.IOLoop.current()
s_iothread = threading.Thread(target=s_ioloop.start)
s_iothread.start()

i = 0
while True:
    ptable.update( [{ 'foo' : i, 'key' : str(i) }] )
    i += 1
    if i > 2:
        i = 0

    time.sleep(.25)

s_iothread.join()

<!DOCTYPE html>
<html>

<head>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, minimum-scale=1, user-scalable=no">

    <script type="text/javascript" src="https://unpkg.com/@finos/perspective/dist/umd/perspective.js"></script>
    <script type="text/javascript" src="https://unpkg.com/@finos/perspective-viewer/dist/umd/perspective-viewer.js"></script>
    <script type="text/javascript" src="https://unpkg.com/@finos/perspective-viewer-datagrid/dist/umd/perspective-viewer-datagrid.js"></script>
    <script type="text/javascript" src="https://unpkg.com/@finos/perspective-viewer-d3fc/dist/umd/perspective-viewer-d3fc.js"></script>

    <link rel='stylesheet' href="https://unpkg.com/@finos/perspective-viewer/dist/umd/material.dark.css">
    <style>
        perspective-viewer{position:absolute;top:0;left:0;right:0;bottom:0;}
    </style>
</head>
<body>
    <perspective-viewer id="viewer" columns='["key", "angle", "raians" ]'>
    </perspective-viewer>
    <script>
        window.addEventListener('WebComponentsReady', async function() {
            const viewer = document.getElementById('viewer');

            // Create a client that expects a Perspective server to accept connections at the specified URL.
            const websocket = perspective.websocket("ws://prdhfalgorsh02:7678/websocket");
            const table = websocket.open_table('my_table');
            viewer.load(table);
            viewer.toggleConfig();
        });
    </script>
</body>
</html>