Development Process and Branching Model

[hyperthunk]

In terms of naming branches, I would like to keep master as one of the mainlines so I'd propose either:

Production: stable Development: master

Or as an alternative, we could use the master branch for production and have something like this:

Production: master Development: development

I think I prefer the latter but I'm open to having a conversation about it.

In either case, issues should be labelled with the appropriate tag so we can differentiate. Once we've written up the merge/rebase procedures for this, we should be able to work on things concurrently without stomping on one another's changes because

@rodlogic - if we can come up with a version of this scheme that works for us both and assign issues to ourselves, then as long as we're able to split up the work so we don't clash too often them I'm willing to set you up as a collaborator and grant you commit access to this project. If you'd prefer to continue working in a fork via pull requests, that's absolutely fine as well.

@edsko - would it make sense to move this project into the haskell-distributed organisation? Github organisations make it easier to manage collaborators and configure access rights, and I'm comfortable opening it up to that wider community (which looks like mostly well-typed folks?) if you think that's appropriate.

[edsko]

@edsko - would it make sense to move this project into the haskell-distributed organisation? Github organisations make it easier to manage collaborators and configure access rights, and I'm comfortable opening it up to that wider community (which looks like mostly well-typed folks?) if you think that's appropriate.

Sure, if everybody agrees with that it's fine with me.

[rodlogic]

if we can come up with a version of this scheme that works for us both and assign issues to ourselves, then as long as we're able to split up the work so we don't clash too often them I'm willing to set you up as a collaborator and grant you commit access to this project. If you'd prefer to continue working in a fork via pull requests, that's absolutely fine as well.

That would be great, but it would be more sensible first to get me in line with the workflow we adopt here. Once clean pull requests are flowing from my end we can re-discuss.

[rodlogic]

Sure, if everybody agrees with that it's fine with me.

Works for me, if that makes sense for whoever owns the haskell-distributed organization.

[edsko]

Works for me, if that makes sense for whoever owns the haskell-distributed organization.

That would be Well-Typed, and we're okay with it. Let me know.

[hyperthunk]

That would be Well-Typed, and we're okay with it. Let me know.

Ok yes please @edsko. I'll probably have to migrate the issue tracker data by hand (urgh) but that's something I can slog through tonight after work.

That would be great, but it would be more sensible first to get me in line with the workflow we adopt here. Once clean > pull requests are flowing from my end we can re-discuss.

Ok cool, I'm going to write up a contributors guide this evening. I will include information about the branch/merge model we're going to adopt in that.

[edsko]

I think that if you give me admin rights to this repository, I can move the entire repository to the haskell-distributed organization (including issues), rather than forking it.

[hyperthunk]

I'm not sure that's really true, but I have added you as a collaborator nonetheless. Let me know if it works! :)

[edsko]

You have given me push rights, but not admin rights. If you go to this repository, if you click on admin, in the danger zone there is a "move this repository to another user or organization". But you can't do that because you're not an admin on haskell-distributed, and I don't even get to see that admin tab on haskell-distributed-platform because I'm not an admin there.

[edsko]

Let me know if you prefer I just fork.

[hyperthunk]

I've moved the repo to my own org and given you admin rights so maybe try again?

[edsko]

Yup, I can do it now. Happy for me to make the transfer now?

[hyperthunk]

Yes let's go for it - this way the issues will be transferred across which works nicely for me.

[rodlogic]

I see the repo moved nicely and the fork automatically points to the new location. It seems that the only real change required in my local fork is to update the upstream url in my .git/config. Anything else?

[hyperthunk]

Nope, that's all you need to do.

[hyperthunk]

@edsko - I'm no longer a repo admin - is that something you can fix?

In terms of using git-flow I'm going to switch to the following scheme:

Production: master Development: development

[hyperthunk]

Erm @edsko I don't seem to have commit rights any more either. Eek! :/

[hyperthunk]

Ok so @rodlogic

Ok cool, I'm going to write up a contributors guide this evening. I will include information about the branch/merge model we're going to adopt in that.

I have mostly written this and will add it to the wiki (and put some notes into the README) as soon as @edsko restores my admin/commit rights.

[edsko]

Erm @edsko I don't seem to have commit rights any more either. Eek! :/

Mwahahahaahhaha :-D

[edsko]

I don't seem to have the rights to give you rights.. I will investigate and fix it :)

[hyperthunk]

Hehehe - ok fair enough.

[hyperthunk]

@rodlogic

That would be great, but it would be more sensible first to get me in line with the workflow we adopt here. Once clean pull requests are flowing from my end we can re-discuss.

Ok cool, I'm going to write up a contributors guide this evening. I will include information about the branch/merge model we're going to adopt in that.

Well, I can't do a thing right now as @edsko isn't an organisation admin and I'm therefore stuck. I've forked (back to) my hyperthunk account for the time being, so I can continue working.

I've added a sketch of a contributors guide on the wiki back there: https://github.com/hyperthunk/distributed-process-platform/wiki/Contributing. Let me know if that is

1. helpful and instructive
2. readable, sensible, etc

If there's anything wrong with it at all, please shout as I'd prefer not to put people off from contributing once we move over to this org officially but at the same time I know how much effort can go into managing contributions and if people know up front what to do it makes it easier for everyone.

@edsko - same question to you with regards the contributor guide.

[rodlogic]

@hyperthunk A few comments (I may have more later):

• the git workflow is quite similar to what I was following lately with one difference: I wasn't merging the feature branch back to my local master branch before issuing the pull request.
• I often see pull requests being used to converge into a final, agreed implementation: i.e. it goes through code reviews, comments and possibly multiple commits until it is ready to be accepted. So, although, a single, squashed commit is always goal for a given iteration, I think we shouldnt see it as a strict rule. The most important is to make sure the commits are rebased and up-to-date with master before a final merge (merge conflicts are not the responsibility of the maintainer).
• Is it a consensus that rebases are better than merges in git's best practices (real question)? If so, I think there was a way to automatically have git use rebase when git pull (?).
• I have been using stylish-haskell to automate reformatting: i.e. stripping trailing white spaces, replacing tabs with spaces, aligning and sorting imports (so no manual labor here) and pragmas, and performing some light weight reformatting of the code. I am not married to one style over another and the guidelines you wrote down/referenced above works for me, but it would be great if we could automate this as much as possible. Unless there are better tools I am not aware of, I would suggest using stylish-haskell and adding a .stylish-haskell.yaml to the root of the repo so we share the same settings.
• Although it adds a bit of labor (I wish tools such as stylish-haskell could automatically do this for me), I find it quite important to be explicit about the specific types being imported from modules. So import Control.Applicative (Applicative) instead of a import Control.Applicative as it become quite obvious where the type is coming from.

[hyperthunk]

Hi @rodlogic - thanks for the feedback.

the git workflow is quite similar to what I was following lately with one difference: I wasn't merging the feature branch > back to my local master branch before issuing the pull request.

Well in theory that only matters in the target repository as you're keeping up to date by rebasing upstream/master onto your own fork. So merging your changes back onto your own master is probably the wrong thing to do, as we might make edits whilst merging the pull request, particularly if we're merging manually. You should just git fetch upstream && git rebase upstream/master to get your fork's master branch up to date.

often see pull requests being used to converge into a final, agreed implementation: i.e. it goes through code reviews, comments and possibly multiple commits until it is ready to be accepted. So, although, a single, squashed commit is always goal for a given iteration, I think we shouldnt see it as a strict rule.

I quite agree. Did my contributor guide make it sound like a single squashed commit was required? If so I should probably make it clearer that isn't a script requirement.

The most important is to make sure the commits are rebased and up-to-date with master before a final merge (merge conflicts are not the responsibility of the maintainer).

Exactly.

Is it a consensus that rebases are better than merges in git's best practices (real question)?

The answer is, it depends. When you want to rewrite a branch's history then rebasing is the way to go in general. When we merge, for example, feature and bugfix branches into development/master then we do not want to rewrite history. It is much easier to review the history of the main (dev/master) branches and look at the changes that were integrated and when, across releases, by not rewriting the branch history. Instead, we disable git's default merge behaviour, which is to apply changes using fast forward if possible, by merging with --no-ff so that the disparate histories are maintained and the branch and merge points remain clear.

If so, I think there was a way to automatically have git use rebase when git pull (?).

As I mentioned just a moment ago, git will automatically try to do a fast-forward when merging using its default strategy. So on the whole, merges will often look like rebases if there are no differences between the two branches.

I have been using stylish-haskell to automate reformatting

Yes, I like it in general, but...

I am not married to one style over another and the guidelines you wrote down/referenced above works for me, but > it would be great if we could automate this as much as possible.

The problem I have with stylish-haskell is its formatting of the import statements. This works fine as long as you run it all the time (and remember to run it all the time) but it suffers from several problems

1. the formatting frequently breaks the 80 column width limit
2. every contributor has to install and use it, or maintain the formatting by hand

Forcing contributors to use a particular tool, especially a code formatting tool, seems a bit heavy handed. If the formatting that stylish-haskell chose for imports wasn't so compilcated, then this wouldn't matter, but it is hard to maintain that scheme by hand.

As an alternative, the formatting scheme I've recommended should never require any manual intervention, as new lines do not have to be aligned with their surroundings and when importing specific entities, the default indentation is being used just as you would for a function or whatever. I can't really see the overheads there, whereas there is an overhead in using stylish-haskell, even if it's just remember to run it - for those who aren't using haskell-mode or vim with this integrated for code style.

Unless there are better tools I am not aware of, I would suggest using stylish-haskell and adding a .stylish-haskell.yaml to the root of the repo so we share the same settings.

If we are going to do this, then just putting in the top of the repository isn't enough. We need to make sure that any contributor is able to use it, so we'd have to do a bit more:

1. have a rule in the makefile that installs stylish-haskell if it's not found on the path (using cabal)
2. have a make rule that commits changes after running stylish-haskell commit: tidy as well.
3. document in the contributors guide that the need to either run make tidy before committing or make commit

I am willing to consider this, but I'd like to hear from @edsko what he thinks as personally I find the import layout that it uses quite horrid and unfamiliar. Can the way that imports are aligned be customised at all? That would make me more included to standardise on this.

Although it adds a bit of labor (I wish tools such as stylish-haskell could automatically do this for me), I find it quite > important to be explicit about the specific types being imported from modules.

I'm a bit lazy about this sometimes, but actually I completely agree with you that this is a good idea most of the time. Until you've imported more than 70% of the types in the module at which point it seems a bit moot.

You could always use eclipse with FP/Haskell support. It automatically adds imports for you and tends towards using specific ones rather than importing eveything. It will not do the same layout as stylish-haskell though. You may be able to get a combination of haskell-mode and scion to do this - I can't quite remember whether I had that level of automation sorted in emacs before, but since I've moved to the 2012 haskell platform most of the really nice emacs integration (scion, ghc-mode) seems too broken to be of much use.

[hyperthunk]

1. have a rule in the makefile that installs stylish-haskell if it's not found on the path (using cabal)
2. have a make rule that commits changes after running stylish-haskell commit: tidy as well.
3. document in the contributors guide that the need to either run make tidy before committing or make commit

Oh and I should point out that this will become a bigger overhead as soon as you encounter someone who is trying to do make tidy and for some reason or other the automated (or manual) install of stylish-haskell doesn't work. That is basically another dependency we've got to maintain, outside the cabal build structure, just to make sure people can contribute to the project in a consistent way without having to manually align all the imports.

[edsko]

@hyperthunk -- You should have admin access to the repository now.

[hyperthunk]

@edsko - sweet, I'm in now. Thanks for sorting that.

[edsko]

@hyperthunk -- I've read through your contributing guidelines and they make perfect sense to me. I also completely agree with you re the layout of import statements -- I hate the way that that stylish-haskell does it (and don't use it for that reason). My personal preference is precisely what you suggest.

[rodlogic]

@hyperthunk Re: stylish-haskell: sounds good! I don't have a strong preference and I buy the arguments.

[hyperthunk]

Thanks both of you for your feedback. I'm going to make some small amendments to the contributor guide and publish it onto the wiki. I'll also probably set up github.pages and reference it from there.

We now have 'development' and 'master' branches. The rest of the branching strategy will become obvious as I start merging things.

I'm going to hold off on making a tag for our current 0.1.0 until we've resolved a couple of things.

1. stability of the gen server implementation
2. hiding as much detail as possible

To help with (2) I have already added a top level Platform module from which to export commonly used entities and provide entry points into the most useful APIs and sub systems. Minimising the entities we export will make it much easier to go ahead with changes without having to make major version increments all the time.

In terms of (1) I think that we need to be a little bit more patient to let the gen server API settle down. IMO implementing supervisor on top of it will help us iron out the kinks nicely. I'm hoping that you're still interested in owning that API @rodlogic as you've done most of the work there. Obviously I'll still chip in to get things done.

re @rodlogic

Re: stylish-haskell: sounds good! I don't have a strong preference and I buy the arguments.

Ok well let's leave that alone for now. Maybe I'll find time to hack stylish-haskell and make the layout configurable some time in the future, at which point I'd be happier to integrate it.

[hyperthunk]

Note to all: I will close this issue once we've tagged a 0.1.0 release.