Streamz with websocket not steaming any data

[leahpence]

I am trying to figure out a right way of streaming data using streamz with websocket. My streaming data is loaded using websocket but not streaming anything.

import config
import websocket, json
import pandas as pd
from streamz.dataframe import DataFrame
from streamz import Stream
import time

ticker_list = ["SCHG"]

# Create DataFrame with specified columns and ticker_list as index
df = pd.DataFrame(columns=['Bid', 'Ask', 'Time'], index=ticker_list)

source = Stream()

def update_dataframe(json_message, df):
    ticker_data = json_message[0]
    ticker_symbol = ticker_data['S']
    df.loc[ticker_symbol, 'Bid'] = ticker_data['bp']
    df.loc[ticker_symbol, 'Ask'] = ticker_data['ap']
    df.loc[ticker_symbol, 'Time'] = pd.to_datetime(ticker_data['t'], unit='s').tz_localize('UTC').tz_convert('America/New_York').strftime('%Y-%m-%d %H:%M:%S%z')

    source.emit(df.copy())  # Emit the updated DataFrame to the stream

def on_open(ws):
    print("opened connection")
    # Authentication
    auth_data = {"action":"auth","key": config.API_KEY,"secret": config.SECRET_KEY}

    # Subscribe
    ws.send(json.dumps(auth_data))
    
    listen_message = {"action":"subscribe","quotes":ticker_list}
    ws.send(json.dumps(listen_message))
    print('subscribed')       

def on_message(ws, message):
    json_message = json.loads(message)
    
    # Update the DataFrame and emit it to the stream
    update_dataframe(json_message, df)

def on_close(ws):
    print("closed connection")

def print_result(x):
    print(x)

sink = source.map(print_result)  # Set up a sink to consume and process the emitted data

socket = "wss://stream.data.alpaca.markets/v2/iex"
ws = websocket.WebSocketApp(socket, on_open=on_open, on_message=on_message, on_close=on_close)
ws.run_forever()

[martindurant]

Are you saying that the on_open, on_message and on_close functions do run, but print_result does not?

You might also be interested in Stream.from_websocket which listens for incoming connections. A streamz native (async) client version would look very similar.

[leahpence]

That is correct, I am not getting any data results window with streamz. Perhaps I am missing something on the way I use steamz code..... When I am using python without streamz code all works good and print normal.

[martindurant]

I am afraid I don't know. You are not using any async streamz stuff here, so the simplified code is essentially

source = Stream()
sink = source.map(print_result)
source.emit(df) # repeated for different df

which does indeed print.

[leahpence]

Does Streamz work on Visual Studio Code.. or just in Jupiter Notebooks?

[martindurant]

The auto-updating visual output is a jupyter-widget thing I don't know if VSCode shows those properly or not. streamz plotting via hvplot can work in a variety of browser platforms, including a standalone webapp; but, again, I don't know what VSCode does.

However, streamz's internals will continue to work anywhere, including a simple terminal, so if output is to a file, printed, or some other side-effect, those will work anyway.