[DP-12] Move logging infrastructure out of the core

[hyperthunk]

[Imported from JIRA. Reported by Tim Watson [Administrator] @hyperthunk) as DP-12 on 2012-12-17 12:19:35]

I would propose that we provide the basic infrastructure to register logging handlers in distributed-process-platform and build on that to add additional capabilities. Another option is to create a new distributed-process-logging package, but I'd really like to build this on top of gen-server/gen-event stuff we're building into -platform personally.

The only problem I can see with that approach is for example with SimpleLocalNet which replaces the logger process on its slaves with one that relays the messages back to the master. Erlang does something similar in its slave module by redirecting all stdout back to the master.

The obvious solution I can see to this, is to go ahead and move the logging infrastructure, then take the master/slave stuff out of SimpleLocalNet and put it into another package that depends on SimpleLocalNet and on -platform. That's still not quite a clean API design though, because there's an implicit dependency on a specific backend which seems unusual to me.

If someone can come up with an alternative that still allows backends to provide this kind of functionality without creating a tangled mess of dependencies then I'm all ears. Currently the backends do not seem to depend on distributed-process at all, which makes sense. I can't see how we'd do CH logging without the Process monad so I'm questioning whether it really makes sense for backends to have this capability at all?

Perhaps this requires some thought about the packaging requirements for CH and how we go about making it easy for developers to work with.

[hyperthunk]

BTW - I'm quite happy to pick up the grunt work on this once we've made some decisions about how it should work.

[edsko]

I'm perfectly happy to for you to take stuff out of SimpleLocalnet. It's a quick-and-dirty backend to get people going, it should not influence design decisions.

[hyperthunk]

I'm perfectly happy to for you to take stuff out of SimpleLocalnet. It's a quick-and-dirty backend to get people going, it > should not influence design decisions.

Cool, so what remains to be decided is whether we're comfortable taking away the ability of backends to override the logging. If that is the case, then we can simply strip things out of the core library (and simplelocalnet) and then open up the relevant ticket(s) in distributed-process-platform.

If we do decide to do that, then I wonder whether we could/should consider what the role of startServiceProcesses should be. There's an open ticket in -platform to look at the OTP Application concept. The role of an application controller is very tightly integrated into the behaviour of an Erlang node, but I'm inclined to keep that in -platform and remove the 'service process' concept from CH altogether.

[hyperthunk]

We should also consider the impact that removing say, the built-in logger process and the start{Slave,Master} APIs from SimpleLocalNet will have on example code and blog posts. That should be factored in, even though it shouldn't necessarily be the basis for decision making either.

[edsko]

I would say ideally logging is independent of the backend.

[hyperthunk]

I would say ideally logging is independent of the backend.

That's my opinion too. Unless @dcoutts and/or @simonmar have major objections then, I suggest we go ahead with this change (in a branch for now, of course!) and then before closing this issue, we will need to

1. remove the logging relay code for start{Slave, Master} from SimpleLocalNet
2. break the say API
3. remove the serviceProcess code from Node.hs
4. put something in the forthcoming release notes
5. notify the parallel-haskell mailing list

Then after we've set up a new logging API, we can pull start{Slave, Master} out of SimpleLocalNet and re-implement the master/slave logging relay on top of the new API in -platform.

[simonmar]

@hyperthunk the reason I've been making changes and getting involved in the development here is that I'm writing a chapter about distributed-process for the O'Reilly book on Parallel and Concurrent Haskell. The first thing I encountered with my examples was that say messages got lost, so I fixed that.

I agree that the right thing to do is to move the logging functionality out of d-p and simplelocalnet. However, my main concern is that I don't have to completely rewrite the examples - as long as it is easy to get hold of say from somewhere, and have it just work, that will be fine with me. We have a few weeks before the book is due to be ready, so you have until then to decide on the API :)

[hyperthunk]

We have a few weeks before the book is due to be ready, so you have until then to decide on the API :)

He he he - ok no problem. I'm going to suggest to the guys that we either make master the stable/live branch or use it as the development branch and branch from the last release (tag) to create a line called 'live' or 'stable'. Any API breaking changes (i.e., anything that's not a bugfix) will get merged into 'master' (or development if we go that way instead) and so you'll still have say sticking around for many months to come before we do the next major release.

In the future, we'll provide say in distributed-process-platform, though obviously there will be a bit more available in that API too but we'll keep the basic log this message to whatever default (i.e., console) setup is available). So *even then hopefully your examples will continue to work, though obviously at that stage the -platform library will need to be installed too.

Really looking forward to the new book BTW! :D

[hyperthunk]

So, I've come to the conclusion that say should remain, as I think any logging infrastructure that does get built up will reside in d-p-platform. We should have some kind of basic logging support in the core d-p library though, and say and its registered logger process is good enough for that. It's also possible to configure the new trace API to write to the logger process, which is quite handy.