Upgrading Brokers 5.5.2 -> 6.0.0, if Zookeepers are on the same nodes, confluent_kafka_2.12 was removed during zookeeper upgrade

[aglahe]

Describe the issue Following the upgrade docs, it appears that after upgrading zookeeper, if the broker is on the same node as a zookeeper....when trying to set an ansible fact, it cannot, since the zookeeper upgrade yml removed the confluent_kafka_2.12 packages.

This may be related to Issue #421 and its PR #422

From the upgrade_broker.yml: - name: Set Current Package Version set_fact: kafka_broker_current_version: "{{ ansible_facts.packages['confluent-kafka-2.12'][0]['version'] }}" when: not confluent_server_enabled|bool

The kafka_broker_current_version can't be set, since the upgrade_zookeeper.yml removed that confluent_kafka_2.12 package.

To Reproduce

1. git clone cp-ansible
2. git checkout 5.5.2-post
3. Use cp-ansible to 'install' 5.5.x zookeeper and brokers
4. git checkout 6.0.0-post
5. upgrade zookeeper: ansible-playbook -i hosts.yml upgrade_zookeeper.yml
6. upgrade brokers: ansible-playbook -i hosts.yml upgrade_kafka_broker.yml -e kafka_broker_upgrade_start_version=5.5.2

Expected behaviour If a node/instance runs both a zookeeper and a broker service, after upgrading the zookeeper, brokers should be upgraded.

Inventory File Nothing crazy...can reproduce using simple inventory file.

Logs TASK [Set Current Package Version] *************************************************************************************************************************************************************************************** Monday 05 October 2020 12:40:04 -0400 (0:00:00.257) 0:00:45.044 ******** fatal: [x.y.z]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'confluent-kafka-2.12'\n\nThe error appears to be in '/Users/aarong/dev/projects/cp-ansible/upgrade_kafka_broker.yml': line 71, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Set Current Package Version\n ^ here\n"}

Environment (please complete the following information):

• OS: Centos 7.7
• CP-Ansible Branch: 5.5.2-post & 6.0.0-post
• Ansible Version Ansible 2.9.0

[JumaX]

Hey @aglahe Thanks for the report. We are actively looking into this and will update soon.

[domenicbove]

I am working on some docs for this. The general approach is to manually stop all services on a given host and then upgrade it using the all.yml playbook ie:

ansible -i /path/to/hosts.yml -m shell -a "systemctl stop confluent-*" host1
ansible-playbook -i /path/to/hosts.yml all.yml --limit host1

You'll want to target kafka controller last, which can be queried with:

ansible -i /path/to/hosts.yml kafka_broker -m import_role -a "name=confluent.kafka_broker tasks_from=dynamic_groups.yml"

[utkarsh5474]

Hi, As per my understanding, this has been fixed. LMK if that's not the case.