Oracle Openstack 3.0.1: OpenStack Re-deployment Ceph OSD Error " ** ERROR: osd init failed: (1) Operation not permitted"

(Doc ID 2317692.1)

Last updated on NOVEMBER 14, 2017

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 3.0.1 and later
Information in this document applies to any platform.

Symptoms

 Your multiple-host Oracle Openstack deployment finished with "Success", however when you inspect container's status with "docker ps", you see Ceph OSD container is still restarting

# docker ps | grep osd
b9ad801f0a9c container-registry.oracle.com/openstack/ol-openstack-ceph-osd:3.0.1 "kolla_start" 2 hours ago Restarting (1) 42 minutes ago ceph_osd_2
#

In container's log file you see below error,

INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Removing existing destination: /etc/ceph/ceph.conf
INFO:__main__:Copying /var/lib/kolla/config_files/ceph.conf to /etc/ceph/ceph.conf
INFO:__main__:Setting permissions for /etc/ceph/ceph.conf
INFO:__main__:Removing existing destination: /etc/ceph/ceph.client.admin.keyring
INFO:__main__:Copying /var/lib/kolla/config_files/ceph.client.admin.keyring to /etc/ceph/ceph.client.admin.keyring
INFO:__main__:Setting permissions for /etc/ceph/ceph.client.admin.keyring
INFO:__main__:Writing out command to execute
Running command: '/usr/bin/ceph-osd -f -d --public-addr 192.168.252.101 --cluster-addr 192.168.252.101 -i 2 --osd-journal /dev/xvdd2 -k /var/lib/ceph/osd/ceph-2/keyring'
2017-10-15 13:04:00.530362 7ff4d109c800 0 ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374), process ceph-osd, pid 17639
starting osd.2 at :/0 osd_data /var/lib/ceph/osd/ceph-2 /dev/xvdd2
2017-10-15 13:04:00.530712 7ff4d109c800 0 pidfile_write: ignore empty --pid-file
2017-10-15 13:04:00.554730 7ff4d109c800 0 filestore(/var/lib/ceph/osd/ceph-2) backend xfs (magic 0x58465342)
2017-10-15 13:04:00.554956 7ff4d109c800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2017-10-15 13:04:00.554962 7ff4d109c800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
2017-10-15 13:04:00.554978 7ff4d109c800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: splice is supported
2017-10-15 13:04:00.636760 7ff4d109c800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2017-10-15 13:04:00.638205 7ff4d109c800 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_feature: extsize is disabled by conf
2017-10-15 13:04:00.710770 7ff4d109c800 1 leveldb: Recovering log #263
2017-10-15 13:04:01.011117 7ff4d109c800 1 leveldb: Delete type=0 #263
2017-10-15 13:04:01.011203 7ff4d109c800 1 leveldb: Delete type=3 #262
2017-10-15 13:04:01.011535 7ff4d109c800 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
HDIO_DRIVE_CMD(identify) failed: Invalid argument
2017-10-15 13:04:03.788942 7ff4d109c800 1 journal _open /dev/xvdd2 fd 17: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
HDIO_DRIVE_CMD(identify) failed: Invalid argument
2017-10-15 13:04:03.871423 7ff4d109c800 1 journal _open /dev/xvdd2 fd 17: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
2017-10-15 13:04:03.872378 7ff4d109c800 1 filestore(/var/lib/ceph/osd/ceph-2) upgrade
2017-10-15 13:04:03.872709 7ff4d109c800 0 <cls> cls/cephfs/cls_cephfs.cc:202: loading cephfs_size_scan
2017-10-15 13:04:03.872809 7ff4d109c800 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
2017-10-15 13:04:03.877375 7ff4d109c800 0 osd.2 22 crush map has features 2200130813952, adjusting msgr requires for clients
2017-10-15 13:04:03.877379 7ff4d109c800 0 osd.2 22 crush map has features 2200130813952 was 8705, adjusting msgr requires for mons
2017-10-15 13:04:03.877382 7ff4d109c800 0 osd.2 22 crush map has features 2200130813952, adjusting msgr requires for osds
2017-10-15 13:04:03.880623 7ff4d109c800 0 osd.2 22 load_pgs
2017-10-15 13:04:03.917360 7ff4d109c800 0 osd.2 22 load_pgs opened 192 pgs
2017-10-15 13:04:03.917376 7ff4d109c800 0 osd.2 22 using 0 op queue with priority op cut off at 64.
2017-10-15 13:04:03.917765 7ff4d109c800 -1 osd.2 22 log_to_monitors {default=true}
2017-10-15 13:04:03.919821 7ff4d109c800 1 journal close /dev/xvdd2
2017-10-15 13:04:04.085313 7ff4d109c800 -1 ** ERROR: osd init failed: (1) Operation not permitted

 

 

Changes

You came across other problems/failures during previous failed deployment attempts.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms