My Oracle Support Banner

Starting up a failed data node causes UP A FAILED DATA NODE CAUSES "A FOREIGN KEY CONSTRAINT FAILS" (Doc ID 1912930.1)

Last updated on SEPTEMBER 23, 2020

Applies to:

MySQL Cluster - Version 7.3 to 7.5 [Release 7.3 to 7.5]
Information in this document applies to any platform.

Symptoms

Error log:

com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (Unknown error code)

Sequence of events:

  1. Node 4 shuts down
  2. Few minutes later the Node 4 is manually started back up
  3. As soon as Node 4 starts the FK problems appear
  4. So stopped Node 4 and confirm the problem is no longer present when Node 4 is shut down
  5. Tried starting Node 4 and this causes the problem to start occurring again
  6. Restart Node 4 with --initial hoping that this will resolve the problem
  7. After the --initial startup on Node 4 is complete, the problem starts occurring again
  8. Shut Node 4 down again as this is seriously affecting a live website
  9. Currently running our SQL cluster on a single data node (Node 3) because of this issue

Have not restarted Node 3 as it doesn't seem to be causing any issues and haven't restarted the SQL nodes either as both of them work correctly when data Node 4 is shut down.

 

Node 3 Error Log

Time: Sunday 27 April 2014 - 18:56:36
Status:  Temporary error, restart node
Message: Node lost connection to other nodes and can not form a unpartitioned
cluster, please investigate if there are error(s) on other node(s)
(Arbitration error)
Error: 2305
Error data: Arbitrator decided to shutdown this node
Error object: QMGR (Line: 6068) 0x00000002
Program: ndbmtd
Pid: 18466 thr: 0
Version: mysql-5.6.15 ndb-7.3.4
Trace: /<PATH>/ndb_3_trace.log.4 [t1..t7]
***EOM***
Time: Friday 2 May 2014 - 13:38:36
Status: Temporary error, restart node
Message: Node lost connection to other nodes and can not form a unpartitioned
cluster, please investigate if there are error(s) on other node(s)
(Arbitration error)
Error: 2305
Error data: Arbitrator decided to shutdown this node
Error object: QMGR (Line: 6068) 0x00000002
Program: ndbmtd
Pid: 6035 thr: 0
Version: mysql-5.6.15 ndb-7.3.4
Trace: /<PATH>/ndb_3_trace.log.5 [t1..t7]
***EOM***


Node 4 error log

Time: Monday 28 April 2014 - 09:22:25
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error,
programming error or missing error message, please report a bug)
Error: 2341
Error data: DbtcMain.cpp
Error object: DBTC (Line: 18290) 0x00000002
Program: ndbmtd
Pid: 2031 thr: 0
Version: mysql-5.6.15 ndb-7.3.4
Trace: /<PATH>/ndb_4_trace.log.10 [t1..t7]
***EOM***
Time: Friday 2 May 2014 - 13:57:58
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error,
programming error or missing error message, please report a bug)
Error: 2341
Error data: DbtcMain.cpp
Error object: DBTC (Line: 18290) 0x00000002
Program: ndbmtd
Pid: 18583 thr: 0
Version: mysql-5.6.15 ndb-7.3.4
Trace: /<PATH>/ndb_4_trace.log.11 [t1..t7]
***EOM***
Time: Thursday 8 May 2014 - 20:45:10
Status: Temporary error, restart node
Message: Node lost connection to other nodes and can not form a unpartitioned
cluster, please investigate if there are error(s) on other node(s)
(Arbitration error)
Error: 2305
Error data: Arbitrator decided to shutdown this node
Error object: QMGR (Line: 6068) 0x00000002
Program: ndbmtd
Pid: 28013 thr: 0
Version: mysql-5.6.15 ndb-7.3.4
Trace: /<PATH>/ndb_4_trace.log.12 [t1..t7]
***EOM***

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
 Workaround
 Permanent fix
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.