CRSCTL Status Showing Node Listener Offline as "lsnrctl" Executable is 0-byte in Size
(Doc ID 1908907.1)
Last updated on JUNE 25, 2023
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.3 and laterOracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Database Exadata Cloud Machine - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Information in this document applies to any platform.
Symptoms
- crsctl status resource -t shows node listener is status offline
ora.LISTENER.lsnr
ONLINE OFFLINE Node01
ONLINE ONLINE Node02
ONLINE OFFLINE Node01
ONLINE ONLINE Node02
- ps -ef |grep tnslsnr showing listener process is running
Node01 /home/grid> ps -ef |grep tnslsnr
grid 2588 2062 0 16:15 pts/0 00:00:00 grep tnslsnr
grid 10173 1 0 May29 ? 08:02:20 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid 2588 2062 0 16:15 pts/0 00:00:00 grep tnslsnr
grid 10173 1 0 May29 ? 08:02:20 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
- And the listener log shows listener is accepting connections
17-JUL-2014 12:37:28 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.XX.XXX)(PORT=59225)) * establish * orcl * 0
17-JUL-2014 12:37:29 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.XX.XXX)(PORT=46862)) * establish * orcl * 0
17-JUL-2014 12:37:30 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.X.XXX)(PORT=52007)) * establish * orcl * 0
17-JUL-2014 12:37:30 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.X.XXX)(PORT=52008)) * establish * orcl * 0
17-JUL-2014 12:37:29 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.XX.XXX)(PORT=46862)) * establish * orcl * 0
17-JUL-2014 12:37:30 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.X.XXX)(PORT=52007)) * establish * orcl * 0
17-JUL-2014 12:37:30 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=XX.XXX.X.XXX)(PORT=52008)) * establish * orcl * 0
- CRS alert log reports listener check failed
2014-07-16 21:23:21.372
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:23:21.587
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node101/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:23:21.692
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:25:09.262
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:25:09.465
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:23:21.587
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node101/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:23:21.692
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:25:09.262
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
2014-07-16 21:25:09.465
[/u01/app/11.2.0/grid/bin/oraagent.bin(10049)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/Node01/agent/crsd/oraagent_grid/oraagent_grid.log"
agent/crsd/oraagent_grid/oraagent_grid.log file shows
2014-07-16 21:23:21.587: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] LsnrAgent::check {
2014-07-16 21:23:21.587: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] lsnrctl status LISTENER
2014-07-16 21:23:21.587: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ AGFW][3422549760] {1:64277:64053} Agent sending reply for: RESOURCE_START[ora.LISTENER.lsnr node02 1] ID 4098:2270227
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::getCrsHome crsHome /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::execCmd 1 USR_ORA_ENV:ORACLE_BASE=/u01/app/grid oracleHome:/u01/app/11.2.0/grid CrsHome:/u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::getCrsHome crsHome /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment Variables ORACLE_HOME=/u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment Variables TNS_ADMIN=/u01/app/11.2.0/grid/network/admin/
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment variable from USR_ORA_ENV ORACLE_BASE=/u01/app/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils:execCmd action = 3 flags = 38 ohome = (null) cmdname = lsnrctl.
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] (:CLSN00010:)Utils:execCmd scls_process_join() uret 209
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] execCmd ret = 209
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] (:CLSN00010:)
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] return val (clsagfwret) = 1
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] LsnrAgent::check }
2014-07-16 21:23:21.587: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] lsnrctl status LISTENER
2014-07-16 21:23:21.587: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ AGFW][3422549760] {1:64277:64053} Agent sending reply for: RESOURCE_START[ora.LISTENER.lsnr node02 1] ID 4098:2270227
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::getCrsHome crsHome /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::execCmd 1 USR_ORA_ENV:ORACLE_BASE=/u01/app/grid oracleHome:/u01/app/11.2.0/grid CrsHome:/u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils::getCrsHome crsHome /u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment Variables ORACLE_HOME=/u01/app/11.2.0/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment Variables TNS_ADMIN=/u01/app/11.2.0/grid/network/admin/
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Adding Environment variable from USR_ORA_ENV ORACLE_BASE=/u01/app/grid
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] Utils:execCmd action = 3 flags = 38 ohome = (null) cmdname = lsnrctl.
2014-07-16 21:23:21.588: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] (:CLSN00010:)Utils:execCmd scls_process_join() uret 209
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] execCmd ret = 209
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] (:CLSN00010:)
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] return val (clsagfwret) = 1
2014-07-16 21:23:21.692: [ora.LISTENER.lsnr][939521792] {1:64277:64053} [check] LsnrAgent::check }
- Manually running Listener control (lsnrctl) to check the node listener process (tnslsnr) from OS prompt, it returns to the prompt with no errors reported. This is for all commands, ie lsnrctl status, lsnrctl services, etc.
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |
| References |