Many Connections From oraagent.bin to ASM or Database Instance (Likely ORA-00020) (Doc ID 1287496.1)

Last updated on NOVEMBER 04, 2014

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 to 11.2.0.2 [Release 11.2]
Information in this document applies to any platform.

Symptoms

11gR2 Grid Infrastructure, observed many connections from oraagent.bin to ASM or database instance, v$session.program shows many "oraagent.bin@racnode1.us.oracle.com (TNS V1-V3)" sessions, and "ps -ef| grep beq" shows many processes:

oracle     823     1  0 Mar20 ?        00:00:00 oracle+ASM21 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))


Process state dump shows similar like the following:

Call stack: ..read<-ntpfprd<-nsbasic_brc<-nsbrecv<-nioqrc..

      O/S info: user: oracle, term: , ospid: 21686
      machine: racnode1.us.oracle.com program: oraagent.bin@racnode1.us.oracle.com (TNS V1-V3)
      application name: oraagent.bin@racnode1.us.oracle.com (TNS V1-V3), hash value=12120032
    Current Wait Stack:
     0: waiting for 'SQL*Net message from client'
        driver id=0x62657100, #bytes=0x1, =0x0
        wait_id=103 seq_num=106 snap_id=1
        wait times: snap=21706 min 32 sec, exc=21706 min 32 sec, total=21706 min 32 sec

Eventually ORA-00020 will happen  as oraagent.bin keeps making new connections without closing them:

ORA-00020: maximum number of processes (nn) exceeded


Once it happens, other instance may fail to restart as no new connection can be made to the ORA-00020 instance:

SQL> startup
ORA-03113: end-of-file on communication channel

Errors in file /u01/app/oracle/diag/asm/+asm/+ASM12/trace/+ASM12_lck0_9071.trc  (incident=233838):
ORA-00600: internal error code, arguments: [1617], [30], [1], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM12/incident/incdir_233838/+ASM12_lck0_9071_i233838.trc
..
LCK0 (ospid: 9071): terminating the instance due to error 480

Trace shows call stack of: .. kgeasi ksbmsg ksbedcic_one_int ksbedcic_one kclabs ..

The ORA-00600 [1617] error will go away once the ORA-00020 instance is restarted (it could mean a restart of Grid Infrastructure  if OCR/VD is on ASM and ASM is getting ORA-00020)

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms