piker icon indicating copy to clipboard operation
piker copied to clipboard
verified publisher icon pikers

Overlays and multicharts UX: follow ups to #420 (and friends)

Open goodboy opened this issue 3 years ago • 0 comments

The UI follow up to #414, it was originally a PR #420 but ended up being a sequence of smaller (factored) PRs which landed finishing up with #455 📚

Hopefully this is the final core-UX feature you always wanted 🏄🏼

mult-instrument overlayed real-time and historical data feeds with simultaneous interaction and "current symbol" selectable order mode control

To run and test the UI:

piker chart btcusdt.binance ethusdt.binance xmrusdt.binance

This should (currently) spawn a chart with all real-time quote feeds and history overlayed on both the fast and slow chart and boot using the method=loglin_ref_to_curve' overlay method by default.

First draft feature set included:

The first MVP for overlay charts lands in #455 and contained a plethora of precursor PRs including:

  • full multi-fqsn support in our data feed layer with multiple symbols per brokerd feed (#414)
  • new samplerd daemon introduced to conduct singleton sampler step event synchronization across multiple flow-actor processors (#440)
  • mega rework to the pre-graphics ShmArray -> QPainterPath rendering pipeline by adding a formalized IncrementalFormatter layer which allows incremental update (and cache) of source data to pre-graphics formats suitable for minimal conversion to a painter path (#447)
  • internally adjusted Axis sticky label APIs (moved to lower level PlotItem in #448)
  • multi-symbol input to the real-time graphics system(s) supporting first draft of multicharts (#449)
  • epoch-time index alignment of multiple time series and a graphics-layer driven "data viz" sub-sys (#451)
  • drastically simplified L1 labels to enable tight (horizontal) stacking of y-axis co-domains (#452)
  • [x] initial multichart latency tuning (#453) through reworks of the display loop's:
    • maxmin() y-ranging and better support for only draw last uppx pixel column style
    • only update from last framed (by type) tick
    • distribution of throttle rate uniformly over number of input fqsns
    • better Viz API pipeline usage to speed up interaction handlers
    • pre-graphics 1-d data caching and use in only draw last uppx pixel column style updates
  • [x] #455 algorithm which allows sanely viewing multiple time series with co-domains normalized via a returns transform:
    • includes support for incremental yrange update from display loop via inputs to ChartView.interact_graphics_loop()
    • removal of all Qt.Signal stuff from view box handlers instead opting to call our .interact_graphics_cycle() manually from mouse event callbacks
    • better leverage of new Viz update methods for faster pipeline updates from incremental quote driven updates in the .ui._display loop.
    • support for multiple overlay methods as finalized in this commit: 
      method: Literal[
    'loglin_ref_to_curve',
    'loglin_ref_to_first',
    'mxmn',

] = 'loglin_ref_to_curve',

Init basic funtionality / usage:

  • [x] basic initial usage is to start a chart with multiple input symbols: piker chart btcusdt.binance ethusdt.binance xmrusdt.binance
  • [x] overlay method control for how to align overlays to one another.
  • [x] L1 labels can be simplified
    • [x] we decided to make them overlay on their respective y-axis
  • [x] how should last datum graphics be offset
    • [x] currently all last datums are time step aligned.

History view

  • [x] curves need to be epoch time aligned instead shm index aligned since obviously it will result in non-time aligned data sets as it is now using the 'index' basis..
  • [x] fix pg.LinearRegion item to show all background overlay curves transparently.

Graphics loop

  • [x] use single Feed.open_multi_steam() for the update task and have it look up appropriate graphics update apis per symbol-quote
  • [x] refactor a bunch of the charting/graphics apis to mostly to remove the global-ish ChartPlotWidget instance and API calls, instead moving all this state into Flow / Flume items and possibly add a Viz type to replace the Flow and instead make the Flow a table of associated src data -> procesor pipelines?
    • more to write up as individual subtasks here

Search UI/UX:

  • [x] make search results and caching work with multi-symbol selections by the GodWidget
  • [x] all selected symbols should be shown in fqsn form in the window title bar

Follow up ToDo task list after the last PR (#455) lands above:

These include outstanding todo features which are still in discussion / research.

Charting graphics layer: flows mgmt, backendy controls, viewlists, schema:

(Edit-able) list of rt-feeds on chart: the ViewList:

  • [ ] more or less very similar to the legacy idea of loading and saving watchlists but instead is a more sophisticated abstraction: a list of trade-able asset pairs that maps to a display of rt-data-flows shown overlayed in a single chart view
  • [ ] obviusly a declarative syntax much like you'd expect for a legacy watchlist but with support for additional metadata: [{'btcusdt.binance: {'pin_to': 'qqq.nasdaq.ib'}}, 'qqq.nasdaq.ib'] or something or other.. (we need to figure out the generic data-structure syntax form(s)
  • [ ] a way to edit multiple lists within the chart UI using (probably) vi-like controls similar to those in the ranger file manager except for moving entries between multiple stacked view lists in the search pane.

bugs^

  • [ ] current cachelist is not in same order as that passed on CLI?

common axes for common dst-asset:

  • [ ] how to support same-dst but cross-broker markets, eg. xbtusdt.kraken and btcusdt.binance
    • [ ] should we use a single axis?
    • [ ] how to determine this automatically?
      • Do a in the same range calc ?
      • have some kinda of adhoc asset name mapping?

OHLC chart overlays:

  • [ ] for ohlc main chart overlays how should we do coloring?
    • currently we pull from a sequence of colors in .ui._display but we should likely make this configurable, possibly through the UI, definitely through a config file.
  • [ ] how to handle overlaid (levered) derivatives where the %returns are substantially larger due to the lever profile? screenshot-2023-03-03_17-44-41
    • [ ] it might make sense in these cases to instead use a "leverage and delta adjusted" scalar for the opt contract so that you can see a "somewhat linear" translation of the deriv price vs. the underlying?
      • the issue is that the call option is clearly causing the bitcoin move to be mega squashed in the y-range.
  • [x] L1 labels were simplified (we decided to make them stack on respective y-axis):
    • [ ] only show (humanized) size on bid/ask not price?
  • [ ] in terms of last datum bars, could they be placed in some kind of no-exactly-aligned (reverse) order of the y-axis from left-to-right?
  • [ ] des-synced last bar issue on 1Min chart seemingly due to startup races where there are N > 2 overlay curves from diff providers?
    • [ ] is this something we can fix in the .data.feed shm allocation logic?
    • [ ] screenshot-2023-03-06_10-47-57

FSP (subcharts)

  • [ ] should fsps like vlm which require a subchart be overlayed or only the currently selected shown by default?
    • [ ] we can make this configurable and maybe offer a sidepane drop down showing all combos and/or a toggle set of symbols from the overlay set?
    • [ ] can we offer a pair selector to make it possible to show the fsp subchart for any market (in the viewlist) not currently selected on the OHLC chart(s)?
      • this would probably help perf issues when using binance :joy:

=> bugs^

  • [ ] we still have some strange issue on our (vlm) FSP charts where when multiple overlays are being backfilled the chart will get all mucked up with strange lines from offscreen connecting to every step? (TODO: PUT PIC HERE from @guilledk!)

Widget UIs: panes and controls, search..:

UI for "pair selection"

  • [ ] dynamic management and selection of a current market pair overlay on a chart as per the ViewList idea from above:
    • [ ] (right-click) context menu options to add/remove some chart from/to a diff viewlist?
    • [ ] editing a ViewList:
      • [ ] mouse UX (shift-click-drag-release?) to create a group from a single/flat results tree?
      • [ ] hot keys for adding a new search result to a group instead of a new widget set (maybe ctl-o or ctl-enter on result row?)
      • [ ] API allowing for the currently selected market which would activate functionality for the order mode and possibly (or independently) the current FSP processing (in subchart).
    • [ ] current market-feed selection controls:
      • [ ] highlight the feed's curve vs. the other overlays
      • [ ] highlight the matching y-axis?
      • [ ] re-order the axes, selected should be closest to curve?
      • [ ] show feed meta-data in sidepane label for selected
      • [ ] allow selection by y-axis?
        • [ ] double click on axis?
      • [ ] or, double click on curve?
      • [ ] or, select by hot key (tab or space global hotkey?)
      • [ ] activate order mode control for that symbol (see more details in section below)

UI for overlay-curve config:

  • [ ] support config (meta) data to choose the pin-method (to .ui.view_mode.overlay_viewlists()) as well as parameters for each mode if supported (eg. 'loglin_ref_to_curve' can take a target curve to pin to)
  • [ ] unify this with the current "feed config" controls: eg. throttle rate, shm buffer sizing, host discovery, what daemon to use as source.. etc?

Search UI/UX:

  • [ ] what viewlist(s) should be loaded by default?
    • [ ] obvs make it configurable
    • [ ] how lazily should we load the rt feed for each?
  • [ ] a new result should be added to existing viewlist?
    • [ ] obvs make this configurable through toml file.
    • [ ] how would this play with the idea of an alt-search for derivatives?
  • [ ] how to open/start a new view list?
  • [ ] how to search through existing viewlists
    • [ ] if you pick a result from one do you load the whole thing?

=> bugs^:

  • [ ] current results order is not the same as CLI input order..

better distinction of pairs on axes?

  • [ ] draw little line from each last label to curve?
  • [ ] some kind of label system to easily know which curve is which feed?
    • [ ] last label should include fqsn?
    • [ ] each y-axis should have fqsn?

Order mode

order mode client multi-market API support:

  • [ ] allow passing a fqsns: list[str] to open_order_mode() and having this setup the client loop and the ems tasks to use feed.open_feed() with the same fqsns passed through?
    • [ ] what should be the handle delivered back? a table of clients or should we add some kind of client-selectable API already that strat code can (eventually) use for easy arbing?

selection oriented UX, show each mkt's EMS state by axis:

  • [ ] pp line arrow-marker should sit on the specific y-axis
    • [ ] depending on "selected symbol" only show that pp?
  • [ ] how/when to show all pps?
    • [ ] show active order summary markers (for those not in view) on each sub-yaxis
    • [ ] maybe only pinned arrow at ppu level always?
  • [ ] support submitting orders only for current selection:
    • [ ] eg. alert submitted should show on only that axis's y-domain
    • [ ] cc cancelling should only cancel current instrument's orders?
    • [ ] tie this in with the viewlist / search result pane, show active orders indicator of some sort?

goodboy avatar Feb 14 '23 15:02 goodboy