Oracle Openstack 3.0: Re-deploy a Ceph Enabled Environment Fails with No JSON Object could be Decoded

(Doc ID 2199721.1)

Last updated on NOVEMBER 23, 2017

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 3.0.1 and later
Information in this document applies to any platform.

Symptoms

After re-partition the CEPH dedicated disks on three Storage Nodes, destroy current deployment, then re-deply will fail with below error:

(snip)
TASK [ceph : Fetching Ceph keyrings] *******************************************
fatal: [o3lr2ctlr2.cn.oracle.com]: FAILED! => {"msg": "The conditional check '{{ (ceph_files_json.stdout | from_json).changed }}' failed. The error was: No JSON object could be decoded", "failed": true}

PLAY RECAP *********************************************************************
o3lr2ctlr1.cn.oracle.com : ok=14 changed=0 unreachable=0 failed=0
o3lr2ctlr2.cn.oracle.com : ok=14 changed=0 unreachable=0 failed=0
o3lr2ctlr3.cn.oracle.com : ok=17 changed=0 unreachable=0 failed=1
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/kollacli/commands/deploy.py", line 87, in take_action
.format(msg=job.get_error_message()))
CommandError: ERROR: Job failed:
Host: o3lr2ctlr2.cn.oracle.com, Task: ceph : Fetching Ceph keyrings, Status: failed, Message: The conditional check '{{ (ceph_files_json.stdout | from_json).changed }}' failed. The error was: No JSON object could be decoded

 

Other similar error can also be found from time to time with same root cause:

(snip)
TASK [ceph : Fetching Ceph keyrings] *******************************************
fatal: [o3lr2ctlr1.cn.oracle.com]: UNREACHABLE! => {"msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", "unreachable": true, "changed": false}

TASK [ceph : Reading json from variable] ***************************************
fatal: [o3lr2ctlr2.cn.oracle.com]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ (ceph_files_json.stdout | from_json) }}): expected string or buffer", "failed": true}

PLAY RECAP *********************************************************************
o3lr2ctlr1.cn.oracle.com : ok=13 changed=4 unreachable=1 failed=0
o3lr2ctlr2.cn.oracle.com : ok=13 changed=4 unreachable=0 failed=1
o3lr2ctlr3.cn.oracle.com : ok=16 changed=2 unreachable=0 failed=1
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/kollacli/commands/deploy.py", line 87, in take_action
.format(msg=job.get_error_message()))
CommandError: ERROR: Job failed:
Host: o3lr2ctlr3.cn.oracle.com, Task: ceph : Generating initial Ceph keyrings and monmap, Status: failed, Message: Container exited with non-zero return code
Host: o3lr2ctlr1.cn.oracle.com, Task: ceph : Fetching Ceph keyrings, Status: unreachable, Message: SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh
Host: o3lr2ctlr2.cn.oracle.com, Task: ceph : Reading json from variable, Status: failed, Message: Unexpected templating type error occurred on ({{ (ceph_files_json.stdout | from_json) }}): expected string or buffer

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms