OLVM: SHE deploy failed ansible_task: Fail if Engine IP is different from engine's he_fqdn resolved IP (Doc ID 2875813.1)

Last updated on JUNE 25, 2022

Applies to:

Symptoms

Fresh hosted-engine setup failed with error:
2022-06-07 15:58:44,265+0700 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u"Fail if Engine IP is different from engine's he_fqdn resolved IP", 'ansible_result': u'type: <type \'dict\'>\nstr: {\'msg\': u"Engine VM IP address is while the engine\'s he_fqdn olvm.oracle.com resolves to 10.10.10.1. If you are using DHCP, check your DHCP reservation configuration", \'changed\': False, \'_ansible_no_log\': False}', 'task_duration': 1, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'}

After SHE deplyment, the OLVM web portal is accessible and https://olvm.oracle.com/ovirt-engine/services/health says:

DB Up!Welcome to Health Status!

But SHE VM is "EngineStarting" status and --check-liveliness says "Hosted Engine is not up!" from self-hosted engine node kvm.oracle.com (KVM host):
# hosted-engine --vm-status

--== Host kvm.oracle.com (id: 1) status ==--

conf_on_shared_storage : True
Status up-to-date : True
Hostname : kvm.oracle.com
Host ID : 1
Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}
Score : 3400
stopped : False
Local maintenance : False
crc32 : 0441ba0e
local_conf_timestamp : 13651
Host timestamp : 13651
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=13651 (Wed Jun 8 17:52:43 2022)
host-id=1
score=3400
vm_conf_refresh_time=13651 (Wed Jun 8 17:52:43 2022)
conf_on_shared_storage=True
maintenance=False
state=EngineStarting
stopped=False
# hosted-engine --check-liveliness
Hosted Engine is not up!

The error message is confusing as:
1. Static IP Address used:
# cat ovirt-hosted-engine-setup-ansible-create_target_vm-xxxxxx.log | egrep "he_vm_ip_addr|he_fqdn" | head -n2
2022-06-07 15:35:46,627+0700 DEBUG var changed: host "localhost" var "he_vm_ip_addr" type "<type 'unicode'>" value: ""10.10.10.1""
2022-06-07 15:35:46,628+0700 DEBUG var changed: host "localhost" var "he_fqdn" type "<type 'unicode'>" value: ""olvm.oracle.com""

2. Same DNS Servers specified for SHE VM as KVM host:
# cat ovirt-hosted-engine-setup-ansible-create_target_vm-xxxxxx.log | grep -A2 "he_dns_addr"
2022-06-07 15:35:46,627+0700 DEBUG var changed: host "localhost" var "he_dns_addr" type "<type 'list'>" value: "[
"10.10.10.11"
]"

# cat /etc/resolv.conf
# Generated by NetworkManager
search oracle.com
nameserver 10.10.10.11

3. Both A record and PTR record of SHE VM are network resolvable:
# host -a olvm.oracle.com
Trying "olvm.oracle.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59773
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;olvm.oracle.com. IN ANY

;; ANSWER SECTION:
olvm.oracle.com. 3600 IN A 10.10.10.1

Received 52 bytes from 10.10.10.11#53 in 5 ms
# host -a 10.10.10.1
Trying "1.10.10.10.in-addr.arpa"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5334
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;1.10.10.10.in-addr.arpa. IN PTR

;; ANSWER SECTION:
1.10.10.10.in-addr.arpa. 3600 IN PTR olvm.oracle.com.

Received 76 bytes from 10.10.10.11#53 in 5 ms

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.