convnetjs
convnetjs copied to clipboard
karpathy
MagicNet for Node.js
Like others I am have been using ConvNetJS in Node.js, server-side.
In order to actually use MagicNet from Node.js given a CSV or similar data, however, one needs to implement preprocessing code like that in automatic.js.
To avoid rewriting such code, in my own experimental branch I separated the preprocessing code out of automatic.js, leaving only the UI layer there. I have tested it and am happy to submit a pull request. But before I do so I want to see if this type of change is wanted and if there any suggestions as far as naming and structure.
Have a look here: https://github.com/bittlingmayer/convnetjs/compare/master...bittlingmayer:csv (I based it off of my fork, but if https://github.com/karpathy/convnetjs/pull/49 is not merged it would be easy to rebase.)
Hey, thank you for opening an issue about this. You're right of course. The whole MagicNet code is weirdly done and not nearly feature-complete enough. I am thinking of removing all of the functionality and the demo from ConvNetJS entirely, because it's probably deceiving or confusing people.
It's also worse: You have to store the exact preprocessor mappings with the model checkpoint so that they can be applied identically to test data. If the preprocessor methods are called on test data then it will generate "garbage" mapping indices and the performance will be random.
Thoughts? If none I'll probably remove MagicNet from ConvNetJS in a few days.
I will leave to you, to decide whether you want this to be a demo, and strictly pedagogic, or a lib on which devs actually rely. Let's assume strictly pedagogic.
It is, I think, really useful to have one Browser Demo that takes arbitrary CSV data. (And making it easy to preload iris.csv and a few similar well-known minisets from diverse domains is instructional too.) But the magic part and associated complications are not necessary for that - better to let performance suffer a bit. In fact, I would think of it as the "simple" "clean" demo (quite the opposite from magic).
In any case such a clean demo will not be very visual, it will be more interesting to play with it programmatically, and that I would say is much better done with Node than inside a browser. And for the HTML page I would put the js inside a text area, so you don't have to make UI controls for each parameter.
I think Ideally I'd like ConvNetJS to be a small core library that devs can rely on. Except I'm just barely familiar with Node - this complicates things :)
In that case I think some good net that requires only a .csv | 2D array | JSON and a label column index would be good.
To be honest I only learnt Node while playing with MagicNet.
(As an aside: consider creating an Org 'convnetjs' ie github.com/convnetjs while it is still available, and then you can decide later to transfer this repo to there. Transferring is easy and all issues history, references from forks etc are preserved.)