Simplify OOD Configuration Files

[Oglopf]

Context

There has been interest in getting OOD to have the configuration files in a singular directory location so users don't get lost in the docs and all the various yaml files they need to setup OOD.

Proposal

One solution proposed is to use one directory with all the various yaml files needed to setup OOD and then symlink those files to where we currently have them sitting

Possible Benefits:

• Single, simple location for every aspect of configuring OOD
• Possible decrease in the friction time to launch OOD
• Prevents users from becoming overwhelmed if they are not a full time sys-admin who now finds themselves managing OOD and Apache.

Possible Problems:

• Would these symlinks ultimately break anything that isn't being thought of
• This would require a reworking of some docs
• Likely would need to examine and expand testing around the ood_portal_generator and other areas of OOD

┆Issue is synchronized with this Asana task by Unito

[rsettlage]

I think the suggestion for symlinks was in the reverse direction. Meaning keep actual files where they are and provide a single directory location with symlinks to the configuration files. This should be non-breaking, require no code change, and is resilient to missing a config file, ie you simply would miss that set of configurations until you located the file and created another symlink. Could be a simple bash script that creates all the symlinks.

[treydock]

Proliferation of symlinks is very frustrating for sys-admins and likely more so for people who are not familiar with finding files. A single directory for configuration files is not ideal. What's better and common practice among Linux software is to have a directory like /etc/my-program and then in there you have like /etc/my-program/config.conf and then overrides can go into /etc/my-program/config.conf.d. Part of this pattern is already in use by OnDemand and it's very powerful and one of the best practices for configuring things on Linux systems.

Right now the configuration logic in many places of OnDemand makes a lot of sense from sysadmin perspective. The config files for most part live in /etc/ood/config. There are subdirectories for distinctly different types of configs like cluster YAML or auto-generated Dex configs. Putting all those files into single directory would not be as intuitive. Also a single configuration file for OnDemand is not practical from not only the size of the file you'd up with but for automation. Managing individual files for automation is far easier than trying to manage one single massive file.

I do not think it's good to simplify things to the point where you alienate sysadmins in favor of novices who might not be as capable of managing complex software stacks. I think good documentation is the best solution for complex software. Over simplifying things too much can reduce the effectiveness of software and lead to issues like inability to effectively manage the software with automation tools.

[treydock]

WRT automation, it's incredibly powerful to be able to do this (notice the purge & recurse)

  file { '/etc/ood/config/clusters.d':
    ensure  => 'directory',
    owner   => 'root',
    group   => 'root',
    mode    => '0755',
    purge   => true,
    recurse => true,
    notify  => Class['openondemand::service'],
  }

That allows the Puppet module to purge unmanaged files which is incredibly useful for automation tools to ensure sane state and avoid configuration drift. It's also good for novice automation users who might not realize that simply deleting something from Puppet doesn't remove the file unless you have something like the above on the parent directory.

Also the complexity to manage large files should be evident here: https://github.com/OSC/puppet-module-openondemand/blob/master/templates/cluster/main.yml.erb

The ood_portal.yml on the other hand is very simple to manage since it's just a basic YAML structure that is built in code and dumped to file. Also it's important for OnDemand to be able to generate some files and Puppet not touch them since they are autogenerated by ood_portal_generator.

[Oglopf]

The proposal was to symlink from a directory to the current files, not sure how to state things clearer but I'll try again.

One large file is very bad, not proposed.

One directory, that holds many files, all symlinked to the current files already used for configuration was the idea.

Hopefully that clears that up.

For Trey, what we are fighting is we have very advanced and knowledgable sys-admins that can use puppet or ansible, but we also have people that are just looped into running OOD and have zero idea of any of what you said above. They'd be lost within one sentence.

Personally, what you say sounds great and I see why we should keep it, but we do have sys-admins that are just handed OOD and have very little idea how to configure things correctly and get a bit lost in all the docs and various files. I think the idea is can we somehow provide a way to do this for these types of institutions.

[treydock]

I'm not sure I follow the benefit of a directory symlinked to many files in other locations. All OnDemand configurations are currently in /etc/ood/config so not sure what could be done differently. Things not in that directory like Apache configs are not intended to be modified by hand but instead modified by ood_portal_generator but if the goal is to make it clear what files get changed, that feels more like a documentation task vs adding symlinks to avoid users thinking they should make manual changes. Maybe there are different configuration files in mind that would be the target of the symlink? What files did you have in mind being the target of the symlinks?

[Oglopf]

Thinking about this last night I'm very inclined to agree with Trey on the current setup.

However, I'm going to leave this issue open as it is tagged as discussion and I am curious to see if any users can provide more information for what they are after with this scheme.