Future project maintenance

[ssbarnea]

@jorisroovers Based on your call for help, I raised this ticket. I think we all could use a good git message linter and I am willing to help.

I had a good track of reviving maintenance on other projects and make them more of a hands-off.

Most of them are hosted under https://github.com/pycontribs/ -- an community effort aimed to help maintaining python projects, mainly by assuring that the projects do not depend on a single (human) point of failure.

The recipe is relatively simple but boring: fully automated CI/CD and bit of housekeeping. Once done most projects can self-sustain.

[jorisroovers]

This is awesome, thank you!

Huge proponent for CI/CD - you're preaching to the choir!

Can you elaborate on what you have in mind?

Here's a quick braindump for more info and context:

• I only really work on gitlint about 2-3 times a year (for about 1-2 weeks at a time) - that almost always results in a release.
• Because of this, I tend to forget some of the details and nuances of the codebase. This is why I've heavily invested in testing and documentation.
• For testing in particular, there's a suite of unit tests, integration tests, pylint and PEP8 checks, as well a basic build test.
• Maybe I overdid the testing a bit, because it's pretty common for minor code changes to require significant test updates. This can be frustrating for both myself as well as new contributors. Then again, quite a few bugs were prevented by single failing unit or integration tests... The testing is what allows me to only work on this occasionally while still be confident everything works.
• In any case, all of these tests run in CI (via github actions) against multiple python and OS versions, see https://github.com/jorisroovers/gitlint/blob/master/.github/workflows/checks.yml
• Since I don't release often, I tend to do some minor additional manual testing, just as a sanity check to make sure everything works.
• The release procedure itself is fully documented on the wiki: https://github.com/jorisroovers/gitlint/wiki/Release-MOP
• Since I don't release often, I've not gotten around to stringing all the steps together into a fully automated release script.
• In addition, a release script that's only run 3 times a year is prone to break or have unexpected side-effects. Hence why I prefer the semi-manual procedure (until now).
• Something I've thought about is pushing a build for every commit/PR to Test PyPi using such a fully automated release script and then just promoting certain builds as production releases on-demand. I think this is what you might be hinting at as well.
• In all of this, what's most paramount for me, is that I can keep gitlint as a hobby (i.e. only work on it occasionally). Practically that means that I don't want to be forced to fire-fight because of a bug being pushed to end-users that requires an urgent fix during the periods I'm not actively working on gitlint.

Thanks again for opening this ticket. I'm excited!

[ssbarnea]

@jorisroovers I will be back later with more details but for start you may explain me why https://github.com/jorisroovers/gitlint/pull/133 didn't trigger any build?! When I seen that I really got scared ;)

[jorisroovers]

I only "recently" (few months ago) moved from TravisCI to GH actions, I reckon this is just an oversight in the CI trigger. Can have a look later if you haven't in the meantime...

[jorisroovers]

Moving discussion here from #133 in which @tbekolay and @sigmavirus24 suggested to moving gitlint under the Python Code Quality Authority project.

@ssbarnea had suggested something similar in the OP, suggesting to move the project under the @pycontribs umbrella.

While I understand neither of those are "official" python projects, I'm honestly flattered at the suggestion alone. Especially PyCQA which has some tools under its umbrella that I've been using for many years, are widely popular and have been inspirations for gitlint. For these reasons I'm leaning more towards PyCQA.

Having said that, I want to make sure I take a bit of time to let this sink in a bit, read up on the details and better understand how it would ultimately impact gitlint and myself - obviously there's some personal attachment to gitlint after working on it for 5 years.

I'm fairly sure that for gitlint it would be a good thing (probably more visibility, more contributors).

[sigmavirus24]

So there's one myth that a lot of people seem to have that I'd like to dispell:

more contributors

I don't think this is strictly true or has any evidence that it's happened for other projects. I don't think it ever happened for Flake8 and I don't know that other projects can say it's happened for them.

That said, it is easier to empower contributors because teams have more granular permissions than I've seen on personal repositories so you could allow folks to triage issues and close them without pushing/merging code. Etc. That's a great way to get folks started, build trust, and help them grow into good maintainers (if you have the time and attention for it).

I want to make sure I take a bit of time to let this sink in a bit

That sounds good to me. Take your time. I know it can be hard to divorce your identity from something you spent so much time on. It's hard to take something your proud of and make it a little less visibly tied to you

[ssbarnea]

@sigmavirus24 While there is no guarantee around "more contributors" part, I do for sure that flake8 is special case. AFAIK, that is the only PyCQA project that is not under github and instead if under gitlab (if i remember correctly for historical reasons as nobody bothered to move it). I am ready to bet that lack of extra contributors on flake8 has something to do with this. At least I am glad that @asottile does a good job maintaining it.

My advice regarding organization: it is not so important as long is not another single person behind it. I seen projects moved multiple times between organizations without much fuss as github seems to be very good at redirects. What it matters is the message sent: this project has better survival chances as there are others that can takeover the burden if needed.

[sigmavirus24]

@ssbarnea other examples on GitHub include baron, red baron, doc8, bandit, and pycodestyle. None have seen a magical increase in contributors since moving to the org

[ssbarnea]

I sent @jorisroovers an invite to pycontribs org, so we could perform the move if we wants. I think that @sigmavirus24 can do the same for PyCQA.

[sigmavirus24]

It seems invites expire (based on recent experience on-boarding the python-modernize team), so I'd rather let Joris decide when I should send the invite

[l0b0]

A bunch of my own and my employer's projects are using gitlint, and I'd be interested in helping out. I'm familiar with the tech (Vagrant, GitHub Actions, Python, static analysis tools), and I'd be interested in especially adding more QA checks like shellcheck, 100% branch coverage, type checks using mypy, Dependabot, pre-commit hooks, possibly Poetry for better package management, and more.

[sigmavirus24]

Wait, Poetry has started supporting real python development? Interesting

Based on your blast of PRs @l0b0 I'm guessing your first order of business would be to change the commit style guide for the project?

[l0b0]

@sigmavirus24 Yes, we're using Poetry for "real" Python development, see for example this repo. I know it might be a controversial subject, so I might bring it up sometime later if people are interested in the whys and hows.

As for the commit style guide, I'm not sure what that is or why I should/would want to modify it? :)

PS: Amazing work on PyCQA by the way! I use a bunch of those tools for my projects, and I'm a big fan of anything which automatically makes my job easier.

[jorisroovers]

Thanks @l0b0 for your interest and all the PRs, very much appreciated! Always fun to hear about who is using gitlint as well :-) Already merged a few PRs and commented on issues.

FYI that it might take me a (long) while to get to everything. That's exactly what this thread is about :-) Please keep submitting PRs etc, if this works well for a while (few months) I'd definitely be willing to add you as a repo collaborator.

---

As for the commit style guide, I'm not sure what that is or why I should/would want to modify it? :)

@sigmavirus24 is referring to the fact that in the PRs you submitted you've been following conventional commits style guide, which I haven't followed before. I don't really have a strong opinion on this for gitlint (vs. a large project with many commits and contributors all working in parallel, where I think this is quite useful).

There's an argument to be made not to enforce it, so we can keep running gitlint running against itself without special config (enforcing Conventional Commit is a contrib rule and not enabled by default). Counter-argument is that it actually show-cases gitlint more if we use a custom configuration against itself 🤷‍♂️ Either way, you can use it, doesn't mean we need to enforce.

---

possibly Poetry for better package management, and more.

I use poetry myself for other projects, and I like it. I've considered this before for gitlint. However, I'm not an expert, have never used it to publish to pypi and not 100% clear on the overall implications. This probably requires a separate issue and more discussion.

---

Other notes:

• I've got some big personal commitments coming up soon, so commitment towards gitlint will be on a low for the rest of this year. That really only means that I'm not making promises on when/if I'll work on gitlint. I just might, time will tell.
• I still haven't decided on whether to move gitlint under PyCQA. Overall, project maintenance seems to be under control. Yes, it's slow, but we're talking about minor fixes and improvements, not major issues or feature gaps. For this reason and the fact that a move is pretty much irreversible, I prefer to keep the project where it is at least for now - that has some clearer recognition benefits for me. Might still take up the offer in the future if it still stands then :-)
• Just in case anyone's interested, a while back I wrote a blog on my experience maintaining gitlint.

[wwuck]

Can I throw another suggestion in the mix?

https://jazzband.co/

I don't have any affiliation with https://github.com/jazzband, but it does seem to work well for the https://github.com/jazzband/pip-tools project.

[sigmavirus24]

Jazzband also has lots of abandoned projects there last time I looked. pip-tools and a few others seem to be the exception, not the rule sadly

[ssbarnea]

@sigmavirus24 I think we may be barking at the wrong tree as @jorisroovers already stated he wants to avoid potential dilution of fame by moving under a community driven organization.

I effectively stopped even trying goodies/tools that have a single maintainer as being too risky. Example https://github.com/ymyzk/tox-gh-actions/issues/74 I guess many python developers do remember at least some of the ugly things that shaken the requests library at some point.

I am more than happy to contribute back to projects that are open by nature. I have a saying in my language that with would relatively translate to once bitten, twice shy.

[sigmavirus24]

dilution of fame by moving under a community driven organization.

I think you're conflating Kenneth's ego trips with Joris' pride here based on the rest of your comments. That doesn't feel fair to Joris. I might be forgetting part of this conversation though

[ssbarnea]

True, is not a fair comparison. The only intent was to highlight how bad it can go in some cases. Joris has deserves all our respect and I really loved his blogpost.

[jorisroovers]

Gitlint is a labor of love. I’m both attached to it and am proud of it, I believe that’s a good thing to have in a maintainer 😊 And yes, it’s nice to have my name clearly associated with it; I don’t think that’s a crime.

If I ever get the feeling that I’m really dropping the ball or am ready to move on, I’ll speak up. I do think that day will eventually come, but I don’t know whether that’s months or years out.

As I said last time, I’m happy to give co-maintainer access to this repo to folks after we’ve done a bit of trust building by collaborating. @sigmavirus24 has crossed that threshold for me after consistent involvement commenting on various issues for the last 2 years or so. So Ian let me know if you’re interested.

I love open source and am a strong believer in how it’s a force for good. The last thing I want is give the impression I’m somehow above that or “not open in nature”. This doesn't mean we all have to agree on everything and that every decision needs to be made based of majority opinion or even consensus (case in point #162) - as long as the discussion is public, respectful, and constructive, it counts for me. Hope that makes sense.

Thanks for continuing to care, means a lot!

[sigmavirus24]

I can't promise I'll have more time than you have presently but I'm happy to help. I'm honored you trust me to add me here

[webknjaz]

I’m happy to give co-maintainer access to this repo to folks after we’ve done a bit of trust building by collaborating

@jorisroovers I haven't seen this stated before, but I'd like to point out that having collaborators on repos hosted under personal user profiles has some technical limitations. If you want to retain power but get rid of those limitations, I suggest creating your own GitHub organization and moving the repo there. There's a lot of projects doing this — orgs have a lot more flexibility compared to individual user accounts. You could even call it jorisroovers-org for all I care, but just the fact that it'd be an org would unlock many possibilities. I saw projects adding -dev or -python suffixes to such org names, among others (so you could move the repo to be gitlint-dev/gitlint, GitHub also makes correct redirects for all the content from the old location). Personally, I have many orgs that I use to categorize repos — it's helpful even when I'm the only org member there. I hope this information will be helpful to you :)

[ssbarnea]

Not sure if yet-another-org is the best, but I can see these orgs as potential matches:

• jazzband - already mentioned, has advantage that any Jazz member can step in and help the project. I guess that as author you should be able to keep the owner on that repo (need to check)
• pycontribs - I am one of the admins on it, you can keep full owner rights on the repo
• pycqa which is full of linters, I have doc8 there which was transitioned from openstack. You can also keep owner rights there too.

[jorisroovers]

Quick acknowledgement of your suggestions - thanks :)

If I ever get the feeling that I’m really dropping the ball or am ready to move on, I’ll speak up. I do think that day will eventually come, but I don’t know whether that’s months or years out.

This is still where my mind is at today.

For the time being, I still prefer the project to remain under my personal account for the reasons outlined before. Appreciate your understanding!