My Oracle Support Banner

Oracle Openstack 4.0.1: unable to deploy with Oracle VM Server 3.4.4 if xend service is running (Doc ID 2361325.1)

Last updated on AUGUST 04, 2018

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 4.0.1 and later
Linux x86-64

Symptoms

Deploy Oracle VM Server 3.4.4 compute node fails with error:
$ kollacli deploy
(snip)
TASK [nova : Waiting for nova-compute service up] ******************************
fatal: [o3lr2net1.cn.oracle.com]: FAILED! => {"_ansible_parsed": true, "stderr_lines": ["Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.", "<class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-d4be1db7-c281-40af-9044-5d3e07d3519a)"], "changed": false, "stdout": "", "_ansible_no_log": false, "stdout_lines": [], "_ansible_delegated_vars": {"ansible_host": "o3lr2net1.cn.oracle.com"}, "cmd": ["docker", "exec", "kolla_toolbox", "openstack", "--os-interface", "internal", "--os-auth-url", "http://XXX-ip:35357", "--os-identity-api-version", "3", "--os-project-domain-name", "default", "--os-tenant-name", "admin", "--os-username", "admin", "--os-password", "9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK", "--os-user-domain-name", "default", "compute", "service", "list", "-f", "json", "--service", "nova-compute"], "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": false, "_raw_params": "docker exec kolla_toolbox openstack --os-interface internal --os-auth-url http://XXX-ip:35357 --os-identity-api-version 3 --os-project-domain-name default --os-tenant-name admin --os-username admin --os-password 9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK --os-user-domain-name default compute service list -f json --service nova-compute", "removes": null, "warn": true, "chdir": null}}, "failed": true, "attempts": 20, "stderr": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-d4be1db7-c281-40af-9044-5d3e07d3519a)", "rc": 1, "delta": "0:00:01.301908", "end": "2018-02-11 07:01:33.558245", "start": "2018-02-11 07:01:32.256337"}

PLAY RECAP *********************************************************************
o3lr2ctlr1.cn.oracle.com : ok=60 changed=12 unreachable=0 failed=0 ignored=0
o3lr2ovs1.cn.oracle.com : ok=56 changed=13 unreachable=0 failed=0 ignored=0
o3lr2kvm1.cn.oracle.com : ok=136 changed=0 unreachable=0 failed=0 ignored=0
o3lr2net1.cn.oracle.com : ok=195 changed=45 unreachable=0 failed=1 ignored=0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/kollacli/commands/deploy.py", line 103, in take_action
.format(msg=job.get_error_message()))
CommandError: ERROR: Job failed:
Host: o3lr2net1.cn.oracle.com, Task: nova : Waiting for nova-compute service up, Status: failed, Message: {"_ansible_parsed": true, "stderr_lines": ["Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.", "<class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-d4be1db7-c281-40af-9044-5d3e07d3519a)"], "changed": false, "stdout": "", "_ansible_no_log": false, "stdout_lines": [], "_ansible_delegated_vars": {"ansible_host": "o3lr2net1.cn.oracle.com"}, "cmd": ["docker", "exec", "kolla_toolbox", "openstack", "--os-interface", "internal", "--os-auth-url", "http://XXX-ip:35357", "--os-identity-api-version", "3", "--os-project-domain-name", "default", "--os-tenant-name", "admin", "--os-username", "admin", "--os-password", "9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK", "--os-user-domain-name", "default", "compute", "service", "list", "-f", "json", "--service", "nova-compute"], "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": false, "_raw_params": "docker exec kolla_toolbox openstack --os-interface internal --os-auth-url http://XXX-ip:35357 --os-identity-api-version 3 --os-project-domain-name default --os-tenant-name admin --os-username admin --os-password 9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK --os-user-domain-name default compute service list -f json --service nova-compute", "removes": null, "warn": true, "chdir": null}}, "failed": true, "attempts": 20, "stderr": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-d4be1db7-c281-40af-9044-5d3e07d3519a)", "rc": 1, "delta": "0:00:01.301908", "end": "2018-02-11 07:01:33.558245", "start": "2018-02-11 07:01:32.256337"}

In this case, nova-compute container keeps restarting:
[root@o3lr2ovs1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9e1cdfefd3e0 oracle/ol-openstack-nova-compute:4.0.1 "kolla_start" 13 minutes ago Restarting (0) 1 seconds ago nova_compute
0202a268be84 oracle/ol-openstack-nova-ssh:4.0.1 "kolla_start" 14 minutes ago Up 11 seconds nova_ssh
764f29daeffe oracle/ol-openstack-multipathd:4.0.1 "kolla_start" 31 minutes ago Up 9 seconds multipathd
846857afb612 oracle/ol-openstack-iscsid:4.0.1 "kolla_start" 31 minutes ago Up 9 seconds iscsid
15886b60de43 oracle/ol-openstack-chrony:4.0.1 "kolla_start" 33 minutes ago Up 11 seconds chrony
f75276e46a76 oracle/ol-openstack-cron:4.0.1 "kolla_start" 33 minutes ago Up 10 seconds cron
0f7f7d638b2e oracle/ol-openstack-kolla-toolbox:4.0.1 "kolla_start" 33 minutes ago Up 9 seconds kolla_toolbox
3c1d5202157c oracle/ol-openstack-fluentd:4.0.1 "kolla_start" 33 minutes ago Up 11 seconds fluentd

Or error as below might be seen:
$ kollacli deploy
(snip)
TASK [nova : Waiting for nova-compute service up] ******************************
fatal: [o3lr2ctlr1.cn.oracle.com]: FAILED! => {"_ansible_parsed": true, "stderr_lines": [], "changed": false, "stdout": "[]", "_ansible_no_log": false, "stdout_lines": ["[]"], "_ansible_delegated_vars": {"ansible_host": "o3lr2ctlr1.cn.oracle.com"}, "cmd": ["docker", "exec", "kolla_toolbox", "openstack", "--os-interface", "internal", "--os-auth-url", "http://XXX-ip:35357", "--os-identity-api-version", "3", "--os-project-domain-name", "default", "--os-tenant-name", "admin", "--os-username", "admin", "--os-password", "9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK", "--os-user-domain-name", "default", "compute", "service", "list", "-f", "json", "--service", "nova-compute"], "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": false, "_raw_params": "docker exec kolla_toolbox openstack --os-interface internal --os-auth-url http://XXX-ip:35357 --os-identity-api-version 3 --os-project-domain-name default --os-tenant-name admin --os-username admin --os-password 9iMYjYQrSFAVlctY26gwOITiSciaN238ZOmHl0RK --os-user-domain-name default compute service list -f json --service nova-compute", "removes": null, "warn": true, "chdir": null}}, "failed": true, "attempts": 20, "stderr": "", "rc": 0, "delta": "0:00:01.805295", "end": "2018-02-10 17:54:52.108451", "start": "2018-02-10 17:54:50.303156"}

PLAY RECAP *********************************************************************
o3lr2ctlr1.cn.oracle.com : ok=225 changed=84 unreachable=0 failed=1 ignored=0
o3lr2ovs1.cn.oracle.com : ok=58 changed=14 unreachable=0 failed=0 ignored=0
o3lr2kvm1.cn.oracle.com : ok=136 changed=0 unreachable=0 failed=0 ignored=0
o3lr2net1.cn.oracle.com : ok=167 changed=53 unreachable=0 failed=0 ignored=0
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/kollacli/commands/deploy.py", line 103, in take_action
.format(msg=job.get_error_message()))
CommandError: ERROR: Job failed:
Host: o3lr2ctlr1.cn.oracle.com, Task: nova : Waiting for nova-compute service up, Status: failed, Message: []

Here nova-compute container is not running at all:
[root@o3lr2ovs1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0202a268be84 oracle/ol-openstack-nova-ssh:4.0.1 "kolla_start" 31 minutes ago Up 4 minutes nova_ssh
764f29daeffe oracle/ol-openstack-multipathd:4.0.1 "kolla_start" 49 minutes ago Up 4 minutes multipathd
846857afb612 oracle/ol-openstack-iscsid:4.0.1 "kolla_start" 49 minutes ago Up 4 minutes iscsid
15886b60de43 oracle/ol-openstack-chrony:4.0.1 "kolla_start" 50 minutes ago Up 4 minutes chrony
f75276e46a76 oracle/ol-openstack-cron:4.0.1 "kolla_start" 50 minutes ago Up 4 minutes cron
0f7f7d638b2e oracle/ol-openstack-kolla-toolbox:4.0.1 "kolla_start" 50 minutes ago Up 4 minutes kolla_toolbox
3c1d5202157c oracle/ol-openstack-fluentd:4.0.1 "kolla_start" 50 minutes ago Up 4 minutes fluentd

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.