neurallayer
Charts should use an custom MemoryJournal to get the timeseries from
Hey :)
Currently, when we want to get a chart of price/trade/signals etc, we need to set the feed / timeframe / assets each time and all the feeds are played for each chart you want to plot.
For exemple, currently we need to do
PriceChart(feed, asset)
SignalChart(feed, strategy)
and this constructor call play the feed two times.
I think it could be more convenient to do something like
run( feed, strategy, chartJournal )
PriceChart(asset, chartJournal )
SignalChart(chartJournal)
where chartJournal is a subclass of MemoryJournal, that define all the metrics that are usefull for the charts. PriceChart for example could get the time series defines by the PriceMetric for the desired asset SignalChart for example could get all the timeseries signals from a SignalMetric.
By doing that, you could play an unique run for your desire strategy / feed / assets and rely on the journal to get the data.
Do you think it could be a good thing, or did i missed something? I could work on that if you think so
I figured that out while trying to create a new roboquant module called roboquant-kandy, that uses kotlin-dataframe and kandy to plot without any single line of HTML / JS
Thank you !!
Not sure it is a trivial refactoring. I think a normal 'Journal' interface could do the job (not sure you need a MemoryJournal), BUT:
Right now, only during the rendering (getOption()) the data is calculated and returned as an Option object. That means that in jupyter notebooks no data is kept in memory after the rendering and plotting many charts doesn't cost much memory.
override fun getOption(): Option {
val data = signalsToSeriesData()
...
Of course at the expense of CPU, since every time you plot, the data is recalculated (aka the feed is replayed).
Would like to keep this behavior, for example for smaller public hosted notebook environments.
Ok, thanks for your answer and i understand that it's done like this for memory reasons. I faced limitations in the charting capabilities of the framework, such as combine lines in the same chart ( price / signal for example), i read the roboquant-charts and roboquant-jupyter to see how i could achieve this on my own.
It's not obvious for users that there are several entry points for playing the feeds, and when your feed grow up and your strategy also grow up, it became a CPU bottleneck (for me at least 😄 ), to replay it for each chart to display.
As you prefer to keep the actual behaviour for small notebook environments, i keep this idea for my own fork.
For your info , here is my naive implementation of pricebarchart with dataframe/kandy and usage of journal. Here journal is just a MemoryJournal with PriceMetric.
class PriceBarChart(
private val journal: ChartJournal,
private val asset: Asset
) : KandyChart()
{
override fun buildDataFrame(): DataFrame<*> {
val pricesOpen = journal.getMetric("price.OPEN.${asset.symbol}".lowercase())
val pricesClose = journal.getMetric("price.CLOSE.${asset.symbol}".lowercase())
val pricesLow = journal.getMetric("price.LOW.${asset.symbol}".lowercase())
val pricesHigh = journal.getMetric("price.HIGH.${asset.symbol}".lowercase())
fun asMap(ts: TimeSeries): Map<Instant, Double> {
return ts.toList().associate { it.time to it.value }
}
val openMap = asMap(pricesOpen)
val closeMap = asMap(pricesClose)
val lowMap = asMap(pricesLow)
val highMap = asMap(pricesHigh)
val times = (openMap.keys + closeMap.keys + lowMap.keys + highMap.keys)
.toSortedSet()
.toList()
if (times.isEmpty()) {
return dataFrameOf(
"time" to emptyList<Instant>(),
"open" to emptyList<Double>(),
"close" to emptyList<Double>(),
"low" to emptyList<Double>(),
"high" to emptyList<Double>()
)
}
val opens = times.map { openMap[it] ?: Double.NaN }
val closes = times.map { closeMap[it] ?: Double.NaN }
val lows = times.map { lowMap[it] ?: Double.NaN }
val highs = times.map { highMap[it] ?: Double.NaN }
return dataFrameOf(
"time" to times,
"open" to opens,
"close" to closes,
"low" to lows,
"high" to highs
)
}
override fun plot(): Plot {
val df = buildDataFrame()
return df.plot {
candlestick(
x = "time",
open = "open",
high="high",
low="low",
close="close"
)
}
}
}
Have a good day !
Thanks, now I understand it better: you want to plot based on recorded metrics in a MemoryJournal.
BTW, code looks nice. How is the chart performance? I did some testing myself at the beginning and found ECharts to be by far the best lib to plot large number of samples. But things could have changed.
Yes exactly, thats what i try to achieve. I figured out that Journal implementations (expecially Memory one) fit exactly for this purpose, because it contains all the timeseries needed to plot and allow me to have one run() for all my plots, in my CPU constrained environnement.
My notebook looks like that :
val feed = AvroFeed("./tradingbotfiles/data_cured/feed/avro/prices/prices.avro")
val journal = ChartJournal( )
val strat = EMACrossover()
org.roboquant.run(
feed= feed,
timeframe = timeframe,
strategy = strat,
journal = journal )
PriceBarChart(
journal = journal,
asset = Stock("AAPL")
).plot()
Concerning youre question regarding perf, i ll give you a feedback further my tests; but for the moment it's pretty convinient to use the Kandy DSL Kotlin-only instead of maintaining large sets of HTML / JS.
Its also very pratical when it comes to add layers to plot, if you want to plot signal/price together :
df.plot {
candlestick(
x = "time",
open = "open",
high="high",
low="low",
close="close"
)
},
points {
x = "time"
y = "signalValue"
}
And of course, having the dataframe with kotlin DSL operations is very nice, even if its not as mature as Python pandas implementation, it helps to close the gap between those two worlds...
Thanks
That allows me to plot those kind of useful charts to see what happen (here price, signals, positions and points for open/close)