Contact al3x and discuss modularizing sovereign

[taoeffect]

Sovereign does way too many things, and because of that, each individual thing it does isn't as good as it could be. If sovereign instead were a "package" system for specific ansible goals (setting up email, setting up calendars, setting up personal dropbox, etc.), then the individual packages would be free to focus and specialize on what they do best.

Ideally:

• You'd be able to run each individual sovereign module independently if that's all you wanted (without using sovereign)
• You'd be able to use sovereign and enable different modules (perhaps via git submodule system) that you wanted.

This would make development easier as well, since Al3x would not be burdened with the responsibility of managing all these different tasks; they could instead be delegated out to experts who focus on those tasks specifically.

cc @PiPeep

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/5845637-contact-al3x-and-discuss-modularizing-sovereign?utm_campaign=plugin&utm_content=tracker%2F8064840&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F8064840&utm_medium=issues&utm_source=github).

[taoeffect]

If @al3x likes this idea, then we could create "official" modules using the sovereign brand, similar to how grunt plugins work. I.e. this could become sovereign-mail, there could be calendar stuff as sovereign-cal, etc.

[al3x]

I'm totally fine with a more modularized Sovereign. It does do quite a lot at this point. Other people have suggested using Ansible Galaxy, Docker, etc. I'm open to whatever as long as the actual work of porting modules gets done :)

[taoeffect]

@al3x Awesome! Glad to hear you're open to the idea! :smile: :+1:

I'll need to look into Ansible Galaxy. I don't recommend Docker (I evaluated it and came to the conclusion that personal servers don't appear to be good use cases for it).

[bgw]

Here's one option:

• Split every role in sovereign into finer-grained roles
  • mailserver might become dovecot and postfix
  • split common into separate fail2ban, unattended-upgrades, etc roles
  • use role dependencies, where that makes sense
• Split every role into a separate repository
  • We can do this with git-subtree to preserve commit history
• Make a few top-level projects that satisfy specific use-cases. These would be responsible for providing variables to roles that makes them work together in a consistent manor.
  • Use git submodules or galaxy to reference the roles
  • sovereign would contain everything
  • empress or maybe sovereign-mail would only include mail-focused roles
  • Maybe an owncloud repository woulc be owncloud focused?
  • etc.
  • The advantage is that for someone who only wants one popular setup, they don't need to trudge through the full sovereign documentation, attempting to figure out safe ways to disable various roles.

Thoughts?

[taoeffect]

So that sounds like you're on the right track. Users need an easy way to say "want sovereign-mail, sovereign-cal, etc."

Git submodules doesn't (to my knowledge) let you pick and choose submodules. It's all-or-nothing as far as I remember. So how do we let them pick and choose (cause it's important to split these things up into separate projects too).

[odinho]

Unconvinced about the need to split into separate git repos. Too much complexity is also a factor. Having everything in one repo Google-style is possible; it can still be perfectly modular. Added cost of a few kilobytes of unused modules are no problem. Also discoverability is higher.

Also, http://debops.org/ seem a bit similar to this effort. Only this seems a bit more focused towards ease of use and one-user, whilst debops is more for helping admins who will use a lot of time on their systems anyway. I stopped with debops because I got a bit overwhelmed.

Maybe it would even be possible to use debops Ansible modules, but build more "ready-built" systems on top of that. Essentially making enduser setup easier by integrating / working at a higher level (just like Sovereign atm).

[taoeffect]

@Velmont Thanks for the link to debops. Very interesting!

I've given this more thought and came up with the following Principle:

Principle: users shouldn't install any more software than is absolutely necessary to get the desired services.

The main concern is modularity and respecting choice for the end-user.

Ideally, we want a system that would allow an interface to be built on top of it with checkboxes for what you want / what you already have installed (Issue #14).

We should shoot for:

• Each service can have multiple implementations (postfix vs. exim, dovecot vs ??, etc.)
• Users might have some of these implementations already installed on their system (like tarsnap) via some other project like sovereign or debops or w/e, so in that case each module (like empress) should default to using those existing service implementations.
• For missing services that are needed, we should choose sane and secure "best defaults" for them, but allow the user to choose alternatives (mariadb vs postgresql, etc.) and then configure the rest of the services to use that system, that database, etc.
• If a user already had a service setup, we should respect it and not do our own version of it (going back to the tarsnap example: if a user already has tarsnap installed, our tarsnap stuff should be skipped).

To re-iterate: if an existing service (like a database) provides the features that are needed for this other service we're trying to install (say, a blog), we should configure the new service to use the existing service if at all possible.

So, how do we bring together these projects in a way that results in the above behavior?

[taoeffect]

Ansible seems to force people to re-invent the wheel if they want to change just one tiny little thing.

We need something that does the opposite: allow people to combine multiple efforts together without stepping on each others toes.

[taoeffect]

We need something with the idea of services: does something exist? If it does, use it. If it doesn't, install it. Right now Ansible overwrites other existing configurations because it assumes it owns them, and that's the problem.

[taoeffect]

@PiPeep suggests splitting up the services in a package (like sovereign or empress) into the smallest possible roles. Then allow the user to specify if they want the package to manage those services itself or to use pre-existing already installed services.

This would allow sovereign and empress (and others) to be combined (if both projects used this approach).

[neuhaus]

We may also need different versions of a component, i.e. a full-fledged mail configuration (with filtering, imap server etc), and - if that's not installed but another module needs to send email - a minimal outbound-only mail system.