htm-community
Implement a C# interface to htm.core
I am thinking that the C# interface should be similar to the Python interface.
- There will be a set of bindings for C# written in C++/cli managed code that are linked with the the core library. This will be a shared library.
- There can be two types of interfaces.
-- Those that directly call the algorithms in the C++ library. For these we will need a C++/cli wrapper for each of the algorithms. -- Those that use the NetworkAPI to access the algorithms. For this we will need one C# wrapper for the NetworkAPI and C# apps would then have access to every algorithm for which a Region wrapper has been written for it, regardless of the language it was written in. -- Any C# modules that implement an algorithm can also implement a NetworkAPI wrapper (which we call a Region). These regions are implemented as a separate plugin. The plugin mechanism would then allow C++ and Python apps to also access the C# algorithms (at least that is the theory).
Let me start by creating a Pull Request and I will add the code I have into it. Then we can discuss the details of our new C# bindings.
@dkeeney is the plan to merge your PR into master, or to continue working on that branch?
Also, down the road when the C# interface is ready, will it be possible to build both the Python interface and the C# interface in the same location? I'm wondering if it makes sense to have 2 separate local copies of the repo if I want to be able to use both. Thanks!
@mrpitts52 welcome to htm.core. I was hoping that you would join us.
is the plan to merge your PR into master,
The way this works is that we clone the repository and check out the branch (or create a new one) for each project. We work on the branch until we are satisfied with the code and it passes all of the unit tests locally. For any new code we must include new documentation (usually comments in code) to explain how it works and new unit tests to prove that it works as advertised. The unit tests are also ran automatically when we push a PR back to gitHub. When it passes all unit tests on all three platforms in the CI (Windows, Linux, OSx), we set the status on the branch to 'ready' in GitHub. Then it is peer reviewed by other developers (probably @breznak and @ctrl-z-9000-times ) and anyone else that is interested. If everyone likes it someone with write permissions can approve it and then we can merge it into the master. At that point it live and is available for everyone to use. Periodically we do a formal release and increment the version number... that is whenever we feel like we need one.
I have about 5 clones of htm.core on my machine, one for each project I am working on but I guess you could do it with one clone and switch between branches. Sometimes I will experiment with several different ways of doing the task and have a different clone for each. Then decide on the one that seems to work the bests...or abandon it altogether.
The current state of this branch is that it will not even compile. It is a bunch of code I wrote up about 1-1/2 years ago when I was thinking about this. It was not complete even then. Since then the library has changed quite a lot..for the better I think. I created a work-in-progress PR so you can have access to the code. We can use this as a starting point for the C# interface or we can throw it all out and start from scratch if you want. In fact, we might want to think about how to break it up into smaller parts so there is less to review at one time.
will it be possible to build both the Python interface and the C# interface in the same location?
We should be able to build Python, C#, and C++ with the same CMake build. However it should also be possible to only build one of the three using option flags for those folks that may have the other environments installed. C# is primarily a Windows thing so Linux or OSx folks may not want it.
@mrcslws I will let you drive this project. I am here to answer questions and explain how the interface needs to work - or you can tell me how you think it should work. If there is a part of it that you do not feel comfortable with you can pass it to me. Once we have a plan, and know what all of the parts are and who is doing what, we can go build it.
You said you were a C# developer. That is cool. Just so I know where you stand;
- Do you feel comfortable with C++/cli ?
- Do you have experience with standard C++ on multiple platforms?
- Do you have any experience with plug-in interfaces?
- Do you have experience with Python?
- About how much time are you expecting to spend. (I am retired and have lots of time but some poor folks have to hold down day jobs and have families to maintain).
@dkeeney that sounds great.
- Do you feel comfortable with C++/cli ?
I actually don't have any prior experience with C++/CLI, but moving between languages hasn't been an issue in the past so it's a challenge I'm happy to tackle.
- Do you have experience with standard C++ on multiple platforms?
This one's a no, the cross platform stuff will be all new to me.
- Do you have any experience with plug-in interfaces?
Not with exactly what appears to be going on here. I do have experience setting up network APIs, so hopefully some of that experience helps.
- Do you have experience with Python?
Some, it's not a language I've used extensively in the past but I have strong backgrounds in several OO languages so I'm not too worried about this.
- About how much time are you expecting to spend
My full time job is unfortunately the top user of my time, along with other random obligations. That said, I'm excited to get into this, so hopefully I can give a couple evenings a week and some good chunks of time on the weekend. I wish I could do more but I'll definitely do what I can.
As mentioned before I have used C# quite a bit and I am very comfortable with git, so at least there a couple things here I do have a background in. If after reading all that you're still willing to let me drive this particular item I'd be happy to do so. :smiley:
I just cloned the repo the other day and got everything building successfully. The readme is great for setup and some info on running test and examples. Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface? A basic overview that indicates what the various 3rd party dependencies are used for and maybe a general guide to what classes do what would be very helpful.
I've been dropped into code bases where I had no idea what was going on before, so I can work my way through the code and figure it out, but I would definitely befenit greatly from an overview like that to speed the process up.
Hi Mike, great to see another soul joining us!
I just cloned the repo the other day and got everything building successfully.
I'm happy the process worked smoothly for you.
Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface?
code-wise, or in generel HTM-wise? Actually, I think we're lacking on the general "what the heck is an HTM and why is it so good then?"-side. Not that we should do all that, but we should like to introductory papers and forum posts to help get people into the picture.
Is there anywhere in the docs that gives kind of a high level architectural overview, both for the core code and the existing Python interface? A basic overview that indicates what the various 3rd party dependencies are used for and maybe a general guide to what classes do what would be very helpful.
this is a good point, thank you! :+1: It would be nice to have an introductory readme for developers wishing to jooin and who need to understand the codebase.
a high level architectural overview
the main classes are
- Spatial pooler (SP)
- Temporal memory (TM)
- then Encoders, and Anomaly
I'd suggest looking at py/htm/examples/*, then src/examples/ (for c++), and then read through the unit tests for select classes of interest, some tests even have a "BasicUsage" example test.
what the various 3rd party dependencies are used for
I don't think that's needed for intro (?), unless you're looking for working on a very specific feature. Here, 3rd party are really only used as dependencies.
maybe a general guide to what classes do
again, the in-class source code doc is very good, but as a document we don't have any.
hope to help somewhat and ask away as you dig into the code :) (maybe start a new issue for the docs)
Ok, as he said, we don't have good overview documentation. But there are some things you can look at to get some idea of what is going on.
@breznak feels that the in-code documentation is adequate. And for figuring out the calling arguments for the algorithms I agree. But for overall understanding I would start with HTM School. Start at the first video and work down through them. It will take a while but it helps a lot in understanding the scope. Jeff Hawkins book On Inteligence is also a good read to put everything in perspective.
Then, there is a set of old documentation for the NuPic API. Parts of this are not accurate. The OPF Framework is no longer supported. The rest of it will give you some introduction to the terms used but we have diverged from this somewhat.
There are the main classes of the algorithms, Spatial Pooler, Temporal memory, Anomaly, and all of the encoders. which were mentioned by @breznak. Those are what the library are all about and everything else are support. For many people this is all they need. They can call the main algorithms from their apps and do whatever they need. In general, @breznak and @ctrl-z-9000-times address those modules.
But there is another set of modules which we now call NetworkAPI. These consist of the Network class and the Region class with its subclasses that make up the plug-ins. This is basically a wrapper around the algorithm classes which orchestrate the execution and data flow for applications. These are the classes that I concentrate on. This is an alternative way to execute the algorithms.
There is also a set of code that make up the C++ to Python interface. Calls actually flow in both directions to make things interesting. There is one set of interfaces to allow python apps to call the algorithms directly. There is another set of interfaces to allow python apps to use the NetworkAPI. The NetworkAPI class will also call back into Python to execute Regions that were written in Python.
The C# project will involve duplicating the Python interface for use by C# applications. Both the algorithm access interface and the NetworkAPI interface. Once those are in place you are free to build wild and wonderful things in C# using the htm.core library.
One of my TODO list items is to re-write the old NuPic API for our library as it is today. Perhaps not all at the calling argument detail but at least much of the overview. I hope someone would show up that has a passion for documentation and can help us out. :-)
Thank you both so much!
The good news is that I have read On Intelligence and I've watched all the HTM School videos, but looks like there is still plenty for me to go through both in the code and over on the HTM Forum. By the way I'm mtopper over there, I like to have different IDs to be unnecessarily confusing.
As I go through the code and inevitably have questions, what's the most logical place to post those? I'm ok with continuing to use this thread but I understand if we'd rather avoid cluttering up an issue thread with a bunch of random questions.
I hope someone would show up that has a passion for documentation and can help us out
@dkeeney I was about to comment that documentation should be the favorite part of every developer's job :laughing: I will open up a new issue for that, someone has to be the downer who brings it up :smiley:
questions, what's the most logical place to post those?
I guess I did not answer that. I normally just use my current issue or PR. If the topic is too far off maybe it is worth adding a new issue.
It might be nice to have an on-going open channel where we can just chat. Like when I like to let everyone know I am going to be traveling the next week, etc. Or when I am trying to figure out what to do next. But we don't have that right now.
@dkeeney @breznak I finally got some free time to dig through the code a bit, and I have a few questions I'm hoping you can help with.
- For this we will need one C# wrapper for the NetworkAPI and C# apps would then have access to every algorithm for which a Region wrapper has been written for it, regardless of the language it was written in.
1.) Let me try to restate this so you can tell me if I'm off base: We already have some 'regions' implemented in Python, which we can do because we have Python bindings for the C++ region building classes. The Network API allows us to expose any number of defined regions, so with a C# wrapper for the Network API, C# apps would have instant access to all existing regions. A couple specific questions on this subject:
- I see the python region classes in the engine_internal project, could you point me to an actual implementation of a region in Python?
- I didn't see a python wrapper for the Network class (based on a search of files with network in the name). Is it called something else, or is this not needed for some reason?
2.) @dkeeney in your PR you noted that your cs_Value class needs to be rewritten since the base Value class changed. I'm guessing by their nature the bindings will always be tightly coupled with the core classes, but is there any way to make them more generic to avoid rework in this sort of scenario? You note that in this case the Value class combined 3 previously existing classes, so it seems like a pretty extreme case, but I'm curious if minor changes also require updates to the bindings.
3.) Looking at the breakdown of the projects, I see the core C++ code is in the LibrarySource project. What is the purpose of the htm_core and htm_core_solo projects?
Outside of these items I 100% understand everything perfectly! Just kidding, but I am making progress, so hopefully I'm not wildly off base with anything above. Thanks!
could you point me to an actual implementation of a region in Python?
c++ (+ py bindings) regions are in src/htm/regions/, there are a few py-only now, see py/htm/advanced/regions/
I didn't see a python wrapper for the Network class
it's in bindings/py/ somewhere, if you grep for Network, you should find it
The regions that are most important are those that are implemented in C++.
See htm.core/src/htm/regions. These are plugins that are automatically registered in RegionImplFactory. The Python and C# regions would need to be registered before they could be used.
network.addRegion( .. ) is the way you identify which regions you want to assign to a network as part of an application.
Got it, and I see that the py_Network implementation is in py_Engine.cpp.
Right now the biggest blind spot for me is the build process, I've never worked with CMake before. I see each project has a CMakeLists file. Are these files manually created or auto-generated?
see each project has a CMakeLists file. Are these files manually created or auto-generated?
cmakelists.txt are written by us, makefiles are then autogenerated. We'll try to help you setting a cmake section for compiling the new code
Are there existing unit tests for the Python bindings? It would be great if we had some to leverage when we add the C# bindings.
Are there existing unit tests for the Python bindings?
yes, the unit tests for the bindings are in htm.core/bindings/py/tests
We also run the full py unit tests: htm.core/py/tests
@dkeeney @breznak Seems like there are two general options for starting off. We could start with the C# network interface, or we could start with the C# bindings for the main algorithm classes. The Network interface would give access to more functionality more quickly for anyone wanting to build a C# app, but unless anyone is really chomping at the bit to do that I think I'm inclined to start with the bindings for the core algorithms first. @dkeeney it looks like this was the approach you were taking based on your PR, correct?
I figure as our first steps we can:
- Add the C# bindings for core algorithms and any necessary helper classes
- Get this all building successfully
- Add C# unit tests to make sure the bindings are working as expected.
Once all that is up and running we can go from there. Thoughts?
it looks like this was the approach you were taking based on your PR, correct?
Yes, but the algorithm API changed quite a lot since I did my original code. We will need a new wrapper for the SDR for one thing.
The NetworkAPI is more stable and much of my original code might be usable but that API is more complicated. It flows in both directions. A C# app can call into the Network object to setup and start a run, but if there are Regions implemented in C# then the Network object needs to be able to allow the C# app to register the Regions and then the Network class needs to be able to call from the C++ back into the C# to execute the regions.
So yes, I like your plan.
Excellent! I can create a new branch for this, then just manually grab things from your PR as we go. Just FYI I am out of town this whole weekend, so probably won't be able to start this in earnest until next week.
@dkeeney Just wanted to provide an update since it's been a while. Unfortunately I haven't had nearly as much free time as I'd like in the last couple weeks, but the time I've had I've reading up on C++/CLI and the whole unmanaged/managed paradigm, as this is my first real foray into that. I have to say the existing code (and the comments) make a lot more sense to me now.
So basically I finally feel ready to get into the actual coding, With it being Thanksgiving week I'm not sure exactly when that will happen but I'll hopefully be bugging you with build questions soon.
Hi Mike, I understand about not finding as much time as expected. That seems to be the way life is. But we will be happy with whatever time you can find. Have a great Thanksgiving week.