Solaris Cluster 3.x Oracle Listener Resource is Failed to Start (Faulted) Because Not Able to Kill Listener Process (Doc ID 1341905.1)

Last updated on NOVEMBER 16, 2016

Applies to:

Solaris Cluster Geographic Edition - Version 3.2 12/06 to 3.3 [Release 3.2 to 3.3]
Solaris Cluster - Version 3.2 12/06 to 3.3 [Release 3.2 to 3.3]
Oracle Solaris on SPARC (64-bit)
Oracle Solaris on SPARC (32-bit)
Oracle Solaris on x86-64 (64-bit)
Oracle Solaris on x86 (32-bit)

Symptoms

Solaris Cluster resource status for Oracle Listener on clustered SMS (Service Management System) shows that the Listener was failed to start (faulted), as shown below:

root@sms01a# clresource status
<snipped>
oracle-listener1-rs sms01a Start failed Faulted
                    sms01b Offline Offline

oracle-listener2-rs sms01b Online Online
                    sms01a Offline Offline
<snipped>


root@sms01a# clrg status

Group Name   Node Name Suspended Status
----------   --------- --------- ------
<snipped>
ora-lsnr1-rg sms01a           No Online faulted
             sms01b           No Offline

ora-lsnr2-rg sms01b           No Online
             sms01a           No Offline
<snipped>


On the example above, the Listener on primary SMS (sms01a) is faulted. But, that is only the cluster monitor saying it thinks the Listener is broken. The "lsnrctl status" command shows that the Listener is running anyway.

oracle@sms01a# lsnrctl status

<snipped>
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
...
Listener Parameter File /u01/app/oracle/product/10.2/network/admin/listener.ora
Listener Log File /u01/app/oracle/product/10.2/network/log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sms01a)(PORT=1521)))
Services Summary...
Service "SMF" has 2 instance(s).
Instance "SMF1", status READY, has 1 handler(s) for this service...
Instance "SMF2", status READY, has 1 handler(s) for this service...

The command completed successfully


sms01a# netstat -an |grep 1521
*.1521 *.* 0 0 49152 0 LISTEN


Compare the process status report of Listener between primary SMS (sms01a) where the Listener cluster resource faulted and secondary SMS (sms01b) where it is online, found that oracle_listener_monitor is not running on the primary SMS, as shown below:

sms01a$ ps -ef |grep -i listener |grep -v grep
oracle 25429 1 0 Apr 25 ? 12:23 /u01/app/oracle/product/10.2/bin/tnslsnr LISTENER -inherit
(missing /opt/SUNWscor/oracle_listener/bin/oracle_listener_monitor as we can see it on sms01b is running)

sms01b
$ ps -ef |grep -i listener |grep -v grep
oracle 15664 1 0 Apr 25 ? 13:04 /u01/app/oracle/product/10.2/bin/tnslsnr LISTENER2 -inherit
root 15790 4686 0 Apr 25 ? 43:56 /usr/bin/ksh /opt/SUNWscor/oracle_listener/bin/oracle_listener_monitor -R oracl



Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms