golearn icon indicating copy to clipboard operation
golearn copied to clipboard

Examples for creating data grids and instances with native types

Open vrecan opened this issue 9 years ago • 3 comments

I am currently trying to take natively typed data in a stream processing system and do predictions on the data. All of the current examples only show how to create instances with csv data. The one example that shows how to create instances only works with converting string data to float64. I already have a map[string]float64 for all of my data I want to put it into an instance and do predictions based on previously learned data.

Any help would be appreciated, if there is no current way to do this I would love to do the work necessary to support this.

vrecan avatar Sep 30 '14 15:09 vrecan

Your options are... limited for the time being. Given Go's typical usage, streaming predictions are an important goal, but for the moment (whilst we get the initial algorithms implemented) we assume a fixed prediction size.

One thing I think might be helpful is to implement a custom FixedDataGrid that's appropriate for your application. I've written a quick wikipage to help you get started. My strategy would be to implement one which:

  • Reports a certain number of Attributes of a certain type in AllAttributes and AllClassAttributes
  • Ignores any calls to AddClassAttribute and RemoveClassAttribute (you set these statically)
  • Holds a fixed amount of data from your streaming source, Get just does some arithmetic.
  • MapOverRows just calls Get using each Attribute for each row in the small amount of stored data.
  • Just returns "Streamed row" for RowString
  • Returns a fixed Size equal to the number of pre-defined Attributes and rows.
  • Implements a getNextBatch function which just fetches another n rows from your data source, and you'd call that after using a predictor.

Don't hesitate to ask any questions! Your experience might be helpful for us as we implement more on-line learning algorithms.

Sentimentron avatar Sep 30 '14 21:09 Sentimentron

Thanks for the help! I will reach out if I have any issues.

vrecan avatar Oct 01 '14 14:10 vrecan

are there any examples of this in github or online?

h4ck3rm1k3 avatar Apr 24 '15 12:04 h4ck3rm1k3