= Anarchy

image::docs/Anarchy.png[Anarchy Diagram,300,300]

== Introducing a Little Anarchy

The world of Kubernetes management is experiencing an Operator revolution.

“The Operator pattern aims to capture the key aim of a human operator who is managing a service or set of services. Human operators who look after specific applications and services have deep knowledge of how the system ought to behave, how to deploy it, and how to react if there are problems.

“People who run workloads on Kubernetes often like to use automation to take care of repeatable tasks. The Operator pattern captures how you can write code to automate a task beyond what Kubernetes itself provides.”

— https://kubernetes.io/docs/concepts/extend-kubernetes/operator/[Kubernetes Operator Motivation^]

Unfortunately, operators are all too often wresting control away from the mere humans who would manage their own resources. Most Operators embed their logic in compiled container images. The typical human systems operator does not program in Go and does not have a container image build pipeline. Human application operators may have experince with builds, but often do not have cluster administrative rights required to deploy operators.

Anarchy democratizes the potential of the Operator pattern by putting the operator logic in the operator configuration. Users write their own logic in Ansible, which requires little more programming experience than using a command line. Through configuration, the Ansible code is directly controlled at runtime to automate anything you can do in the cluster. Anarchy makes your cluster truly self-governing.

== Anarchy in Action

TODO - Add example anarchy operator for managing an application with a Helm chart.

== Architecture

image::docs/AnarchyDiagram.png[Anarchy Diagram,545,131]

Each AnarchySubject is managed by an AnarchyGovernor. Each AnarchyGovernor is configured to run Ansible tasks on a single AnarchyRunner. AnarchySubjects can have AnarchyActions created to schedule task execution. AnarchyRuns are created for specific Ansible task execution, for processing an subject event handler, run an action, or handle an action callback event.

image::docs/AnarchyCallbackDiagram.png[Anarchy Diagram,748,338]

. An action defined in the AnarchyGovernor config is scheduled for an AnarchySubject, creating an AnarchyAction . An AnarchyRun is created to execute the AnarchyAction's tasks . Ansible in the AnarchyRun calls a Job Template or Survey in Ansible Tower, passing the callback URL and token for the AnarchyAction . Ansible executing in Ansible Tower calls the AnarchyAction callback . A new AnarchyRun is created to process the callback from Tower . The AnarchyRun for the callback updates the status of the AnarchySubject

== Installation

=== Prerequisite

Anarchy is using Kopf and needs the kopfpeerings.kopf.dev CustomResourceDefinition must be configured before deploying Anarchy.

oc apply -f kopfpeerings.crd.yaml

=== Install with Helm

Clone this repo to your local workstation and change directory:

git clone https://github.com/redhat-cop/anarchy.git cd anarchy

Install with Helm

helm template anarchy helm/ --include-crds | oc apply -f -

Check your to make sure the anarchy-operator pod is running and the default runner starts soon after:

$ oc get pods NAME READY STATUS RESTARTS AGE anarchy-5f7d7898d7-f6dww 1/1 Running 0 11m anarchy-runner-default-9767c79b6-vt6b4 1/1 Running 0 12m

Anarchy can be installed even if you do not have cluster admin privileges. This will require the CRDs and ClusterRoles to already have been installed.

helm template anarchy helm/ | oc apply -f -

== Anarchy Design

The Anarchy Operator is configured with custom resource type AnarchyGovernor in order to manage AnarchySubject resources. Each AnarchySubject is managed according to a single AnarchyGovernor. The AnarchyGovernor defines actions to perform against APIs to instantiate and manage the AnarchySubject. Each action performed for an AnarchySubject according to the AnarchyGovernor definition is represented as an AnarchyAction custom resource. An AnarchyAction always begins with an call to an API. The Anarchy operator listens for callbacks to its own API for events relating to actions such as notifications that an action has completed, or encountered an error. The AnarchyGovernor defines event handlers for actions which may include scheduling further AnarchyActions to occur for the AnarchySubject.

This repository includes a test suite that demonstrates these capabilities by calling a test API. The usage of the test suite is explained in the "Testing" section below. The conceptual overview of the test design is described here.

Let's start with the AnarchySubject definition:

apiVersion: anarchy.gpte.redhat.com/v1 kind: AnarchySubject metadata: generateName: test- namespace: anarchy-operator spec: desiredState: started <1> governor: test <2> parameters: <3> openshift_release: "4.1" aws_region: us-east-1 repo_version: "3.11" subdomain_base_suffix: .example.opentlc.com

<1> The desired state of the resource, this is an arbitrary string which should be implemented by the AnarchyGovernor. <2> The test AnarchySubject references the name of the AnarchyGovernor that will manage it. <3> Each subject may include a list of parameters to pass to the API, though the governor and API get the final say in how and when the parameters are used.

The test AnarchyGovernor definition is shown here:

apiVersion: anarchy.gpte.redhat.com/v1 kind: AnarchyGovernor metadata: name: test spec:

Ansible processing for this governor will occur on the default runner.

runner: default

var: ansible_tower_hostname: tower.example.com cloud_provider: ec2 varSecrets:

• name: api-creds var: api_creds
• name: aws-credentials

The subjectEventHandlers provide configuration for how to respond to

AnarchySubjects being added, updated, and deleted.

subjectEventHandlers: # The create event is processed only for subjects that are newly created. create: tasks: # The anarchy_subject_update module is provided to make it easy to # update the AnarchySubject relating to the current action. - name: Set state provision-scheduled in subject status anarchy_subject_update: metadata: labels: state: provision-scheduled status: state: provision-scheduled # The anarchy_schedule_action module is used to create AnarchyActions # for the current AnarchySubject. In this case it schedules an # AnarchyAction to be processed immediately. - name: Start Provision anarchy_schedule_action: action: provision

# The `update` event is processed when a resource changes.
update:
  # The `anarchy_subject` variable stores the state of the AnarchySubject
  # which triggered this update. A useful pattern is to implement state
  # handling using `spec.desiredState` and `status.state`.
  - when: >-
      anarchy_subject.spec.desiredState|default('') == 'started' and
      (anarchy_subject.status|default({})).state|default('') == 'stopped'
    block:
    - name: Set state start-scheduled in subject status
      anarchy_subject_update:
        metadata:
          labels:
            state: start-scheduled
        status:
          state: start-scheduled
    - name: Schedule start
      anarchy_schedule_action:
        action: start
  - when: >-
      anarchy_subject.spec.desiredState|default('stopped') == 'stopped' and
      (anarchy_subject.status|default({})).state|default('') == 'started'
    block:
    - name: Set state stop-scheduled in subject status
      anarchy_subject_update:
        metadata:
          labels:
            state: stop-scheduled
        status:
          state: stop-scheduled
    - name: Schedule stop
      anarchy_schedule_action:
        action: stop

# The `delete` event is processed when a subject delete is requsted. This
# is detected by the presence of a `metadata.deletionTimestamp`. This should
# schedule an action that will result in removing the finalizer from the
# subject when complete.
delete:
  tasks:
  - name: Schedule destroy
    anarchy_schedule_action:
      action: destroy

Actions represent entry points for doing something related to a resource.

Each action here consists of an API request followed by callbackHandlers

to respond to callbacks from the API endpoint.

actions: provision: tasks: - name: Call API uri: url: https://{{ ansible_tower_hostname }}/api/v2/job_templates/job-runner/launch/ url_username: "{{ api_creds.user }}" url_password: "{{ api_creds.password }}" validate_certs: false method: POST return_content: true body_format: json body: extra_vars: job_vars: >- {{ anarchy_subject.vars.job_vars | default({}) | combine(anarchy_governor.vars.job_vars, recursive=True) | combine({ 'ACTION': 'provision', 'meta': { 'deployer': {'entry_point': 'ansible/main.yml'}, 'tower': {'action': 'provision'} } }, recursive=True) }} ignore_errors: true

  callbackHandlers:
    started:
      tasks:
      - name: Set state provisioning in subject status
        anarchy_subject_update:
          metadata:
            labels:
              state: provisioning
          status:
            state: provisioning
    - event: complete
      tasks:
      - name: Set state started in subject status
        anarchy_subject_update:
          metadata:
            labels:
              state: started
          status:
            state: started
      # Subsequent actions are scheduled to run later with the `after` parameter.
      - name: Schedule stop
        anarchy_schedule_action:
          action: stop
          after: 8h
      - name: Schedule destroy
        anarchy_schedule_action:
          action: destroy
          after: 6d

stop:
  tasks:
  - name: Call API for stop
    uri:
      url: https://{{ ansible_tower_hostname }}/api/v2/job_templates/job-runner/launch/
      url_username: "{{ api_creds.user }}"
      url_password: "{{ api_creds.password }}"
      validate_certs: false
      method: POST
      return_content: true
      body_format: json
      body:
        extra_vars:
          job_vars: >-
            {{ anarchy_subject.vars.job_vars | default({})
             | combine(anarchy_governor.vars.job_vars, recursive=True)
             | combine({
                 'ACTION': 'stop',
                 '__meta__': {
                   'deployer': {'entry_point': 'ansible/lifecycle.yml'},
                   'tower': {'action': 'stop'}
                 }
               }, recursive=True)
            }}
    ignore_errors: true

  callbackHandlers:
    started:
      tasks:
      - name: Set state stopping in subject status
        anarchy_subject_update:
          spec:
            desiredState: stopped
          metadata:
            labels:
              state: stopping
          status:
            state: stopping
    complete:
      tasks:
      - name: Set state stopped in subject status
        anarchy_subject_update:
          metadata:
            labels:
              state: stopped
          status:
            state: stopped

start:
  tasks:
  - name: Call API
    uri:
      url: https://{{ ansible_tower_hostname }}/api/v2/job_templates/job-runner/launch/
      url_username: "{{ api_creds.user }}"
      url_password: "{{ api_creds.password }}"
      validate_certs: false
      method: POST
      return_content: true
      body_format: json
      body:
        extra_vars:
          job_vars: >-
            {{ anarchy_subject.vars.job_vars | default({})
             | combine(anarchy_governor.vars.job_vars, recursive=True)
             | combine({
                 'ACTION': 'start',
                 '__meta__': {
                   'deployer': {'entry_point': 'ansible/lifecycle.yml'},
                   'tower': {'action': 'start'}
                 }
               }, recursive=True)
            }}
    ignore_errors: true

  callbackHandlers:
    started:
      tasks:
      - name: Set state starting in subject status
        anarchy_subject_update:
          metadata:
            labels:
              state: starting
          status:
            state: starting
    complete:
      tasks:
      - name: Set state started in subject status
        anarchy_subject_update:
          metadata:
            labels:
              state: started
          status:
            state: started
      - name: Schedule stop
        anarchy_schedule_action:
          action: stop
          after: 8h

destroy:
  tasks:
  - name: Call API for destroy
    uri:
      url: https://{{ babylon_tower_hostname }}/api/v2/job_templates/job-runner/launch/
      url_username: "{{ api_creds.user }}"
      url_password: "{{ api_creds.password }}"
      validate_certs: false
      method: POST
      return_content: true
      body_format: json
      body:
        extra_vars:
          job_vars: >-
            {{ anarchy_subject.vars.job_vars | default({})
             | combine(anarchy_governor.vars.job_vars, recursive=True)
             | combine({
                 'ACTION': 'destroy',
                 '__meta__': {
                   'deployer': {'entry_point': 'ansible/destroy.yml'},
                   'tower': {'action': 'destroy'}
                 }
               }, recursive=True)
            }}
    ignore_errors: true

  callbackHandlers:
    complete:
      tasks:
      - name: Delete anarchy subject
        anarchy_subject_delete:
          remove_finalizers: true

== Testing

=== Examples

Examples are found in the examples folder.

== Configuration

Environment valiable to specify how long subjects should remain cached when active: ANARCHY_SUBJECT_CACHE_AGE_LIMIT default 600