Installation of Oracle Clusterware 12c(12cR1 & 12cR2) on SLES 12 fails with CRS-8503 [Signal / Exception: 11] (Doc ID 2270947.1)

Last updated on JULY 25, 2017

Applies to:

Oracle Database - Standard Edition - Version 12.1.0.2 and later
Linux x86-64

Symptoms

Installation of Oracle Clusterware 12.2.0.1 on SLES 12 SP4 fails with below error,when running root.sh

2017-05-09 14:44:51: Starting CSS in exclusive mode
2017-05-09 14:44:51: Configured CRS Home: /u01/app/grid/12.2.0.1
2017-05-09 14:44:51: Executing cmd: /u01/app/grid/12.2.0.1/bin/crsctl start crs -excl -cssonly
2017-05-09 14:44:55: Command output:

> CRS-2672: Try to start "ora.evmd" on "los-oraracn1"
> CRS-2672: Try to start "ora.mdnsd" on "los-oraracn1"
> CRS-2676: Starting "ora.mdnsd" on "los-oraracn1" successfully
> CRS-2676: Starting "ora.evmd" on "los-oraracn1" successfully
> CRS-2672: Try to start "ora.gpnpd" on "los-oraracn1"
> CRS-2676: Starting "ora.gpnpd" on "los-oraracn1" successful
> CRS-2672: Try to start "ora.cssdmonitor" on "los-oraracn1"
> CRS-2672: Try to start "ora.gipcd" on "los-oraracn1"
> CRS-2676: Starting "ora.cssdmonitor" on "los-oraracn1" successfully
> CRS-2674: Starting "ora.gipcd" on "los-oraracn1" not successful
> CRS-2679: Try to clean up "ora.gipcd" on "los-oraracn1"
> CRS-2681: Clean up of "ora.gipcd" to "los-oraracn1" successfully
> CRS-2673: Try to stop "ora.cssdmonitor" on "los-oraracn1"
> CRS-2677: Stopping "ora.cssdmonitor" on "los-oraracn1" successful
> CRS-2673: Try to stop "ora.mdnsd" on "los-oraracn1"
> CRS-2677: Stopping "ora.mdnsd" on "los-oraracn1" successful
> CRS-5804: Error in communication with agent process
> CRS-4000: Start command not successful or completed with errors.
>End Command output

2017-05-09 14:44:55: CRS-2672: Try to start "ora.evmd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2672: Try to start "ora.mdnsd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2676: Starting "ora.mdnsd" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-2676: Starting "ora.evmd" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-2672: Try to start "ora.gpnpd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2676: Starting "ora.gpnpd" on "los-oraracn1" successful
2017-05-09 14:44:55: CRS-2672: Try to start "ora.cssdmonitor" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2672: Try to start "ora.gipcd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2676: Starting "ora.cssdmonitor" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-2674: Starting "ora.gipcd" on "los-oraracn1" failed successfully
2017-05-09 14:44:55: CRS-2679: Try to clean up "ora.gipcd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2681: Clean up "ora.gipcd" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-2673: Try to stop "ora.cssdmonitor" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2677: Stopping "ora.cssdmonitor" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-2673: Try to stop "ora.mdnsd" on "los-oraracn1"
2017-05-09 14:44:55: CRS-2677: Stopping "ora.mdnsd" on "los-oraracn1" successfully
2017-05-09 14:44:55: CRS-5804: Error in communication with agent process
2017-05-09 14:44:55: CRS-4000: Command start not successful or completed with errors.

2017-05-09 14:44:55: The exlusive mode cluster start failed, see Clusterware alert log for more information
2017-05-09 14:44:55: Executing cmd: /u01/app/grid/12.2.0.1/bin/clsecho -p has -f clsrsc -m 119
2017-05-09 14:44:55: Command output:
> CLSRSC-119: Start of the exclusive mode cluster failed

 

From GI Alert.log

2017-05-10 14: 19: 47.702 [CLSECHO (17912)] CLSRSC-0567: Beginning Oracle Grid Infrastructure configuration.
2017-05-10 14: 20: 35.332 [OCRCONFIG (22865)] CRS-2101: OLR has been formatted with version 4.
2017-05-10 14: 21: 17.717 [OHASD (23972)] CRS-8500: The Oracle Clusterware OHASD process starts with the operating system process ID 23972
2017-05-10 14: 21: 17.718 [OHASD (23972)] CRS-0714: Oracle Clusterware Release 12.2.0.1.0.
2017-05-10 14: 21: 17.722 [OHASD (23972)] CRS-2112: OLR service started on node los-oraracn1.
2017-05-10 14: 21: 17.730 [OHASD (23972)] CRS-1301: Oracle High Availability Service started on node los-oraracn1.
2017-05-10 14: 21: 38.953 [CLSECHO (24692)] OKA-0620: OK is not supported on this operating system version: '4.4.49-92.14-default SP2'
2017-05-10 14: 21: 57.605 [OHASD (25699)] CRS-8500: Oracle Clusterware's OHASD process starts with the operating system process ID 25699
2017-05-10 14: 21: 57.606 [OHASD (25699)] CRS-0714: Oracle Clusterware Release 12.2.0.1.0.
2017-05-10 14: 21: 57.611 [OHASD (25699)] CRS-2112: OLR service started on node los-oraracn1.
2017-05-10 14: 21: 57.618 [OHASD (25699)] CRS-1301: Oracle High Availability Service started on node los-oraracn1.
2017-05-10 14: 21: 57.837 [ORAAGENT (25806)] CRS-8500: The ORAAGENT process of the Oracle Clusterware starts with the operating system process ID 25806
2017-05-10 14: 21: 57.854 [CSSDMONITOR (25832)] CRS-8500: The Oracle Clusterware CSSDMONITOR process starts with the operating system process ID 25832
2017-05-10 14: 21: 57.854 [CSSDAGENT (25827)] CRS-8500: Oracle Clusterware's CSSDAGENT process starts with operating system process ID 25827
2017-05-10 14: 21: 57.862 [ORAROOTAGENT (25819)] CRS-8500: The ORAROOTAGENT process of Oracle Clusterware starts with the operating system process ID 25819
2017-05-10 14: 21: 57.898 [ORAROOTAGENT (25819)] CRS-8503: The ORAROOTAGENT Oracle Clusterware process with operating system process ID 25819 received a fatal error or exception code 11.
2017-05-10T14: 21: 58.064744 + 02: 00
Errors in file /u01/app/oracle/diag/crs/los-oraracn1/crs/trace/ohasd_orarootagent_root.trc (incident = 985):
CRS-8503 [Signal / Exception: 11] [Instruction Addr: 0x7f1ec2deb4a0] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
Incident details in: /u01/app/oracle/diag/crs/los-oraracn1/crs/incident/incdir_985/ohasd_orarootagent_root_i985.trc

2017-05-10 14: 21: 58.424 [ORAROOTAGENT (25878)] CRS-8500: The ORAROOTAGENT process of the Oracle Clusterware starts with the operating system process ID 25878
2017-05-10 14: 21: 58.455 [ORAROOTAGENT (25878)] CRS-8503: The ORAROOTAGENT Oracle Clusterware process with operating system process ID 25878 received a fatal error or exception code 11.
2017-05-10T14: 21: 58.489246 + 02: 00
Errors in file /u01/app/oracle/diag/crs/los-oraracn1/crs/trace/ohasd_orarootagent_root.trc (incident = 993):
CRS-8503 [Signal / Exception: 11] [Instruction Addr: 0x7f91731a84a0] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
Incident details in: /u01/app/oracle/diag/crs/los-oraracn1/crs/incident/incdir_993/ohasd_orarootagent_root_i993.trc

2017-05-10 14: 21: 58.839 [ORAROOTAGENT (25904)] CRS-8500: The ORAROOTAGENT process of Oracle Clusterware starts with the operating system process ID 25904
2017-05-10 14: 22: 06.902 [ORAROOTAGENT (25904)] CRS-5822: Agent '/u01/app/grid/12.2.0.1/bin/orarootagent_root' logged off from server. Details at (: CRSAGF00117 :) {0: 11: 3} in /u01/app/oracle/diag/crs/los-oraracn1/crs/trace/ohasd_orarootagent_root.trc.
[OHASD (26860)] CRS-8500: The Oracle Clusterware OHASD process starts with the operating system process ID 26860
2017-05-10 14: 22: 18.879 [OHASD (26860)] CRS-0714: Oracle Clusterware Release 12.2.0.1.0.
2017-05-10 14: 22: 18.883 [OHASD (26860)] CRS-2112: OLR service started on node los-oraracn1.
2017-05-10 14:22:18,891 [OHASD (26860)] CRS-1301: Oracle High Availability Service started on node los-oraracn1.
2017-05-10 14: 22: 19.348 [ORAAGENT (26951)] CRS-8500: The ORAAGENT process of Oracle Clusterware starts with the operating system process ID 26951
2017-05-10 14: 22: 19.363 [CSSDAGENT (26972)] CRS-8500: Oracle Clusterware's CSSDAGENT process starts with operating system process ID 26972
2017-05-10 14: 22: 19.364 [CSSDMONITOR (26977)] CRS-8500: The Oracle Clusterware CSSDMONITOR process starts with the operating system process ID 26977
2017-05-10 14: 22: 19.372 [ORAROOTAGENT (26969)] CRS-8500: The ORAROOTAGENT process of Oracle Clusterware starts with the operating system process ID 26969
[ORAROOTAGENT (26969)] CRS-8503: The Oracle clusterware process ORAROOTAGENT with operating system process ID 26969 received a fatal error or exception code 11.
2017-05-10T14: 22: 19.445790 + 02: 00
Errors in file /u01/app/oracle/diag/crs/los-oraracn1/crs/trace/ohasd_orarootagent_root.trc (incident = 1001):
CRS-8503 [__lll_unlock_elision () + 48] [Signal / Exception: 11] [Instruction Addr: 0x7f132bc6c4a0] [Memory Addr: (n

 

ohasd_orarootagent_root_<incident number>.trc shows the following stack trace-

Dump file /u01/app/oracle/diag/crs/los-oraracn1/crs/incident/incdir_985/ohasd_orarootagent_root_i985.trc
[TOC00000]
Jump to table of contents
Dump continued from file: /u01/app/oracle/diag/crs/los-oraracn1/crs/trace/ohasd_orarootagent_root.trc
[TOC00001]
CRS-8503 [__lll_unlock_elision()+48] [Signal/Exception: 11] [Instruction Addr: 0x7f1ec2deb4a0] [Memory Addr: (nil)] [] [] [] [] [] [] [] []

========= Dump for incident 985 (CRS 8503 [__lll_unlock_elision()+48]) ========
Starting a Diag Context default dump (level=3)
[TOC00003]
----- START Event Driven Actions Dump ----
---- END Event Driven Actions Dump ----
[TOC00003-END]
[TOC00004]
----- START DDE Actions Dump -----
Executing SYNC actions
Executing ASYNC actions
[TOC00005]
----- START DDE Action: 'dumpFrameContext' (Sync) -----
[TOC00006]
----- START Frame Context DUMP -----
Status: 0
---------------------------------------------------------------------------
[0001] (location=FILE:clsdadr.c LINE:4896 ID:dbgec*):
clsdsigSignal pointer=0x7f1ebd3921e0
---------------------------------------------------------------------------
----- END Frame Context DUMP -----
[TOC00006-END]
----- END DDE Action: 'dumpFrameContext' (SUCCESS, 0 csec) -----
[TOC00005-END]
[TOC00007]
----- START DDE Action: 'dumpDiagCtx' (Sync) -----
[TOC00008]
------- Diag Context Dump -------
----- General DiagCtx Dump -----
Serial#: 0
Status: 3
Flags: 0x21D
Usage type: 0
Diag Fixed SGA information:
Status: 2
Flags: 0x0
Incident Flag: UNSET
Heap: 0x7f1ea8001170
Flood Control information:
Initialized: NO
Enabled: NO
Diag init parameters:
Flags: 0x1
Base Flag: 0x0
Options: 0xC0820
ADR option: 1
ADR Base: '/u01/app/oracle'
Product Type: 3
Product ID: 'los-oraracn1'
Instance ID: 'crs'
Flood Control Gate: (nil)
Flood Control Ctx size: 0
ADR default lib ID: 5
UTS in mem trace enabled: TRUE
UTS default bucket size: 4294967295
UTS default bucket: 0x29fda50
Incident ID cache cbf: (nil)
Incident ID cache pop init: FALSE
UTS default trace file attr:
Flags: 0x1
Prefix: 'ohasd_orarootagent_root'
PIDStr: ''
Suffix: ''
Notify cbk: 0x7f1ec79b87fe
Size limits: 18446744073709551615
Size cbk: (nil)
DDE alive update cbk: (nil)
DDE alive start cbk: (nil)
DDE alive clear cbk: (nil)
Custom Flood Control cbk: (nil)
----- END General DiagCtx Dump -----
----- DDE Diagnostic Information Dump -----
Depth: 1
DDE flags: 0x0
Heap: 0x7f1ea8014110
Incident Context pointer in diag: 0x7f1ebd38ae60
Incident ID Cache: 0x7f1ea801dff0 (PGA)
Invocation Context #: 0
----- Invocation Context Dump -----
Address: 0x7f1ea8024110
Phase: 3
flags: 0x10E0000
Incident ID: 985
Error Descriptor: CRS-8503 [__lll_unlock_elision()+48] [Signal/Exception: 11] [Instruction Addr: 0x7f1ec2deb4a0] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
Error class: 0
Problem Key # of args: 1
Number of actions: 9
----- Incident Context Dump -----
Address: 0x7f1ebd38ae60
Incident ID: 985
Problem Key: CRS 8503 [__lll_unlock_elision()+48]
Error: CRS-8503 [__lll_unlock_elision()+48] [Signal/Exception: 11] [Instruction Addr: 0x7f1ec2deb4a0] [Memory Addr: (nil)] [] [] [] [] [] [] [] []

 


 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms