microprediction icon indicating copy to clipboard operation
microprediction copied to clipboard

If you can measure it, consider it predicted

microprediction client, docs, install, examples, slack and book tests deploy

This was supposed to be the microprediction client README. Github made this a "user page". They know best. Well then hi this is my dog. This is my blog. I've always worked in the private sector, though very occasionally publish. My book is out soon (see also /building_an_open_ai_network) and here's a lit of other stuff I've written.

I'm dynamic and fascinating. Not really but I run a slack channel for those interested in time-series, covariance prediction, optimization and other things enabling semi-autonomous collective microprediction. Most Friday's at noon I make myself available to anyone interesting in these things via a regular open google meet (see here). I'm not big on creating other deterministic future work interruptions even though I do realize there is excellent tooling for this.

The TimeMachines, Precise, and HumpDay packages

I maintain three benchmarking packages to help me, and maybe you, surf the open-source wave.

Topic Package Elo ratings Methods Data sources
Univariate time-series timemachines Timeseries Elo ratings Most popular packages (list) microprediction streams
Global derivative-free optimization humpday Optimizer Elo ratings Most popular packages (list) A mix of classic and new objectives
Covariance, precision, correlation precise See notebooks cov and portfolio lists Stocks, electricity etc

These packages aspire to advance online autonomous prediction in a small way, but also help me notice if anyone else does!

The microprediction.org platform

Yes I maintain a live exchange where distributional time-series prediction algorithms (Python, R, Julia mostly) duke it out, trying to predict future values of streams. But after reiterating that there is $50,000 in prediction prizes up for grabs (wow that's incredible!) I'll send you to the better organized documentation that supercedes this README.

No? Okay fine. TLDR:

  • I publish live data repeatedly, like this say, and it creates a stream (like this stream or this one or any from the listing).
  • Algorithms like this guy compete to make distributional predictions 1 min ahead, 5 min ahead, 15 min ahead and 1 hr ahead.

In this way I can:

  • Get live prediction of public data for free (sometimes I make the "public" data obscure)
  • Indirectly benefit from hundreds of packages from Github of uncertain quality, and not just Python.
  • Sip pina coladas while the accuracy magically improves over time. Read about this example.

Should you be interested in helping micropredict stuff? Well if nothing else, I'll point out that due to its obscurity, running this script makes a heck of a lot more economic sense than bitcoin mining - try it out or see the docs.

Just to lean on the differences between this and the other benchmarking efforts, the TimeMachines package is traditional open-source software for point-estimates and confidence, whereas the Microprediction client offers live crowd based distributional prediction and also, in theory, exogenous data search. But the similarity is that hundreds of algorithms compete at Microprediction and quite a few of the TimeMachines algorithms (see /skaters) are involved, drawn from packages like river, pydlm, tbats, pmdarima, statsmodels.tsa, neuralprophet, Facebook Prophet, Uber's orbit, Facebook's greykite and more. Some are open source (look for CODE badges on leaderboards) but others are private to their author.

Memorable Unique Identifiers and other platform repos

  • The muid identifier package is explained in this video.
  • microconventions captures things common to client and server, and may answer many of your more specific questions about prediction horizons, et cetera.
  • rediz contains server side code. For the brave.
  • There are other rats and mice like getjson, runthis and momentum.

Some of my other packages:

  • winning - A recently published fast algorithm for inferring relative ability from win probability.
  • embarrassingly - A speculative approach to robust optimization that sends impure objective functions to optimizers.
  • pandemic - Ornstein-Uhlenbeck epidemic simulation (related paper)
  • firstdown - The repo that aspires to ruin the great game of football. See Wilmott paper.
  • m6 - Illustrates fast numerical rank probability calculations, using winning. However since the rules changed, this isn't that useful for M6 anymore. The precise package is way more useful, and put one person on the podium!

The book is on the way

You can help me choose!

The beginnings of a book website includes issues allowing you to complain about bugs.