automate

automate copied to clipboard

User Story

we're going to need to upgrade to es 7 or 8 by the end of the year, bc 6 will be eol. https://github.com/chef/automate/issues/4165

let's look at the breaking changes and see if there's anything that affects the ingest/config mgmt code

Aha! Link: https://chef.aha.io/epics/SH-E-738

current version: https://github.com/chef/automate/blob/master/components/automate-elasticsearch/habitat/plan.sh#L7

es: https://www.elastic.co/guide/en/elasticsearch/reference/current/es-release-notes.html

theoretically, we should be able to simply update the version in the automate-elasticsearch plan, rebuild, and see the results of upgrading.

plan is to update to ES 7.9

also need to look at plan for bringing customers from A1 (es5) to latest (which will now be es7) see https://chefio.slack.com/archives/C6JED4R8W/p1596221710414500

Elasticsearch breaking changes https://www.elastic.co/guide/en/elasticsearch/reference/7.0/release-notes-7.0.0.html#breaking-7.0.0

Time formats with timezones need updated https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-to-java-time.html#java-time-migration-incompatible-date-formats.

"yyyy-MM-dd'T'HH:mm:ssZ" need to be changed to "yyyy-MM-dd'T'HH:mm:ssX" for times like "2017-09-01-07:58:06Z". But for time like "2010-01-01T01:02:03-07:00" we need to add more X like "yyyy-MM-dd'T'HH:mm:ssXXX"

All "elastic.NewTypeQuery(mappings.DocType)" filters can be removed because there is not a doctype anymore for documents. So All documents in indexes have the same doctype.

All elastic.NewIdsQuery(mappings.DocType) can remove the doctype because all the documents have the same doctypes.

compliance-service reporting test

  1) Failure:
07_suggestions_spec.rb#test_0001_works [/Users/lancewf/workspace/automate/components/compliance-service/api/tests/07_suggestions_spec.rb:141]:

--- expected
+++ actual
@@ -1 +1,3 @@
-[]
+["Apache should be enabled--apache-02--",
+ "Apache should be running--apache-01--",
+ "Apache should start max. 1 root-task--apache-03--"]


1 runs, 20 assertions, 1 failures, 0 errors, 0 skips

11_wonky_profiles_spec.rb#test_0001_works [/Users/lancewf/workspace/automate/components/compliance-service/api/tests/11_wonky_profiles_spec.rb:28]:
Expected /InSpec archive failed for /
 to match "Received RST_STREAM with error code 2"
.

  1) Failure:
07_suggestions_w_filters_spec.rb#test_0001_works [/Users/lancewf/workspace/automate/components/compliance-service/api/tests/07_suggestions_w_filters_spec.rb:50]:
--- expected
+++ actual
@@ -1 +1,10 @@
-["Disable neighbor solicitations to send out per address--sysctl-27--"]
+["Apache should start max. 1 root-task different--apache-03--",
+ "Assign one global unicast IPv6 addresses to each interface--sysctl-28--",
+ "Disable Accept Router Preference from router advertisement--sysctl-22--",
+ "Disable Apache’s follows Symbolic Links for directories in alias.conf--apache-11--",
+ "Disable Core Dumps--sysctl-31--",
+ "Disable Directory Listing for directories in alias.conf--apache-12--",
+ "Disable IPv6 autoconfiguration--sysctl-26--",
+ "Disable IPv6 if it is not needed--sysctl-18--",
+ "Disable Source Routing--sysctl-13--",
+ "Disable TRACE-methods--apache-09--"]

1 runs, 2 assertions, 1 failures, 0 errors, 0 skips

After adding 100 compliance nodes with "chef_load_compliance_nodes 100" the automate-elasticsearch's status is "unknown" When running curl "$ELASTICSEARCH_URL/_cat/indices?pretty" I get the below error message.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "circuit_breaking_exception",
        "reason" : "[parent] Data too large, data for [<http_request>] would be [1582945680/1.4gb], which is larger than the limit of [1478832947/1.3gb], real usage: [1582945680/1.4gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=0/0b, model_inference=0/0b, accounting=1601192/1.5mb]",
        "bytes_wanted" : 1582945680,
        "bytes_limit" : 1478832947,
        "durability" : "PERMANENT"
      }
    ],
    "type" : "circuit_breaking_exception",
    "reason" : "[parent] Data too large, data for [<http_request>] would be [1582945680/1.4gb], which is larger than the limit of [1478832947/1.3gb], real usage: [1582945680/1.4gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=0/0b, model_inference=0/0b, accounting=1601192/1.5mb]",
    "bytes_wanted" : 1582945680,
    "bytes_limit" : 1478832947,
    "durability" : "PERMANENT"
  },
  "status" : 429
}

Because the default number of shards has changed from 5 in es6 to 1 in es7 each time an index or template is created we need to set the number of shards to 5.

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#_static_index_settings https://www.elastic.co/guide/en/elasticsearch/reference/6.8/index-modules.html#_static_index_settings

For the automate-cs-oc-erchef component, the "_all" is not a valid field in ES 7. https://github.com/chef/automate/blob/master/components/automate-cs-oc-erchef/habitat/config/elasticsearch-index-init.json#L18

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/mapping-all-field.html