My Oracle Support Banner

Oracle Openstack 3.0.1: OpenStack Deployment failed at "Host: {HOST}, Task: mysqlcluster : Waiting for bootstrap container, Status: failed, Message: 1" (Doc ID 2317690.1)

Last updated on JUNE 07, 2021

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 3.0.1 to OpenStack 3.0.1 [Release ODOS3]
Oracle Cloud Infrastructure - Version N/A and later
Linux x86-64

Symptoms

 Your multiple-host Oracle OpenStack 3.0.1 deployment failed with below errors in /var/log/kolla/kolla.log,

[2017-09-13 16:25:31,568] INFO kollacli.common.ansible.job TASK [mysqlcluster : Waiting for bootstrap container] **************************
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job fatal: [openstackctr0]: FAILED! => {"cmd": ["docker", "wait", "bootstrap_mysqlcluster"], "end": "2017-09-13 17:26:03.270047", "_ansible_no_log": false, "stdout": "1", "changed": true, "rc": 0, "start": "2017-09-13 16:25:30.992815", "stderr": "", "delta": "1:00:32.277232", "invocation": {"module_name": "command", "module_args": {"warn": true, "executable": null, "_uses_shell": false, "_raw_params": "docker wait bootstrap_mysqlcluster", "removes": null, "creates": null, "chdir": null}}, "stdout_lines": ["1"], "failed_when_result": true, "warnings": [], "failed": true}
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job PLAY RECAP *********************************************************************
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job {CMPHOST}: ok=38 changed=14 unreachable=0 failed=0 ignored=0
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job {CTRNODE}: ok=116 changed=48 unreachable=0 failed=1 ignored=0
[2017-09-13 17:26:03,552] INFO kollacli.common.ansible.job {CTRNODE}: ok=105 changed=45 unreachable=0 failed=0 ignored=0
[2017-09-13 17:26:03,578] ERROR kollacli Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/kollacli/commands/deploy.py", line 103, in take_action
.format(msg=job.get_error_message()))
CommandError: ERROR: Job failed:
Host: openstackctr0, Task: mysqlcluster : Waiting for bootstrap container, Status: failed, Message: 1

 

The mentioned bootstrap_mysql container's log looks like below

# docker logs -f bootstrap_mysqlcluster
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Copying /var/lib/kolla/config_files/mysqlcluster-api.my.cnf to /etc/my.cnf
INFO:__main__:Setting permissions for /etc/my.cnf
INFO:__main__:Writing out command to execute
+++ pre-start mysqld +++
+++ mysqld PID = 10 +++
+++ waiting for mysqld to start +++
2017-10-14 22:42:32 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2017-10-14 22:42:32 0 [Note] /usr/sbin/mysqld (mysqld 5.6.34-ndb-7.4.13-cluster-gpl) starting as process 10 ...
2017-10-14 22:42:32 10 [Note] Plugin 'FEDERATED' is disabled.
2017-10-14 22:42:32 10 [Note] InnoDB: Using atomics to ref count buffer pool pages
2017-10-14 22:42:32 10 [Note] InnoDB: The InnoDB memory heap is disabled
2017-10-14 22:42:32 10 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-10-14 22:42:32 10 [Note] InnoDB: Memory barrier is not used
2017-10-14 22:42:32 10 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-10-14 22:42:32 10 [Note] InnoDB: Using Linux native AIO
2017-10-14 22:42:32 10 [Note] InnoDB: Using CPU crc32 instructions
2017-10-14 22:42:32 10 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2017-10-14 22:42:32 10 [Note] InnoDB: Completed initialization of buffer pool
2017-10-14 22:42:32 10 [Note] InnoDB: Highest supported file format is Barracuda.
2017-10-14 22:42:32 10 [Note] InnoDB: 128 rollback segment(s) are active.
2017-10-14 22:42:32 10 [Note] InnoDB: Waiting for purge to start
2017-10-14 22:42:32 10 [Note] InnoDB: 5.6.34 started; log sequence number 1625987
2017-10-14 22:42:32 10 [Note] NDB: Changed global value of binlog_format from STATEMENT to MIXED
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
+++ waiting for mysqld to start +++
2017-10-14 22:43:03 10 [Warning] NDB: server id set to zero - changes logged to bin log with server id zero will be logged with another server id by slave mysqlds
2017-10-14 22:43:03 10 [Note] NDB Binlog: Starting...
2017-10-14 22:43:03 10 [Note] NDB Binlog: Started
2017-10-14 22:43:03 10 [Note] NDB Binlog: Setting up
2017-10-14 22:43:03 10 [Note] NDB Util: Starting...
2017-10-14 22:43:03 10 [Note] NDB Util: Wait for server start completed
2017-10-14 22:43:03 10 [Note] NDB Index Stat: Starting...
2017-10-14 22:43:03 10 [Note] NDB Index Stat: Wait for server start completed
2017-10-14 22:43:03 10 [Note] NDB Binlog: Created schema Ndb object, reference: 0x0, name: 'Ndb Binlog schema change monitoring'
2017-10-14 22:43:03 10 [Note] NDB Binlog: Created injector Ndb object, reference: 0x0, name: 'Ndb Binlog data change monitoring'
2017-10-14 22:43:03 10 [Note] NDB Binlog: Setup completed
2017-10-14 22:43:03 10 [Note] NDB Binlog: Wait for server start completed
2017-10-14 22:43:03 10 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 0a28182f-b131-11e7-a112-0021f6156c7d.
2017-10-14 22:43:04 10 [Note] Server hostname (bind-address): '{IPADDR}'; port: {PORT}
2017-10-14 22:43:04 10 [Note] - '{IPADDR}' resolves to '{IPADDR}';
2017-10-14 22:43:04 10 [Note] Server socket created on IP: '{IPADDR}'.
2017-10-14 22:43:04 10 [Note] Event Scheduler: Loaded 0 events
2017-10-14 22:43:04 10 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.34-ndb-7.4.13-cluster-gpl' socket: '/var/lib/mysql/mysql.sock' port: {PORT} MySQL Cluster Community Server (GPL)
2017-10-14 22:43:04 10 [Note] NDB Util: Wait for cluster to start
2017-10-14 22:43:04 10 [Note] NDB Index Stat: Wait for cluster to start
2017-10-14 22:43:04 10 [Note] NDB Binlog: Check for incidents
2017-10-14 22:43:04 10 [Note] NDB Binlog: Wait for cluster to start
2017-10-14 23:43:05 10 [Warning] NDB : Tables not available after 3600 seconds. Consider increasing --ndb-wait-setup value

 

If you also check mysqlcluster_mgmt" container's log,  you will see below:

2017-10-14 22:56:07 [ndb_config] ERROR -- at line 47: At least one database node (ndbd) should be defined in config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Removing existing destination: /var/lib/mysql-cluster/config.ini
INFO:__main__:Copying /var/lib/kolla/config_files/mysqlcluster-mgmt.config.ini to /var/lib/mysql-cluster/config.ini
INFO:__main__:Setting permissions for /var/lib/mysql-cluster/config.ini
INFO:__main__:Writing out command to execute
2017-10-14 23:09:47 [ndb_config] ERROR -- at line 47: At least one database node (ndbd) should be defined in config file

 

Changes

You are  deploying your Oracle Openstack cluster in this way,


In host view 

<span class="awizStartDelimiter"><span class="kmfixedwidthfont">&nbsp;<span class="awizEndDelimiter">&nbsp;</span></span>
</span>

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.