Resolve TimesTen Error "Process NNNNNNN still exists, marked zombie"

(Doc ID 1316739.1)

Last updated on MARCH 02, 2016

Applies to:

Oracle TimesTen In-Memory Database - Version 11.2.1 and later
Information in this document applies to any platform.
***Checked for relevance on 17-Dec-2013***

Symptoms


Bug 12376674 involves a condition where a newly connected process is confused with a recently disconnected process, with the result that the wrong process space is reinitialized. The following characteristics were identified for this bug.

(1) Objects are cleaned up for a still-connected session. Example: pid 15061 initially connects here:

2011-03-30 17:16:35.53 Info: : 16794: Got hello from pid 15061, type library (extprocPLSExtProc (LOCAL=NO) )
...
2011-03-30 17:16:35.58 Info: : 16794: 15061 0x5c16f0: ConnectComplete Y N /mydata/ttdb
2011-03-30 17:16:35.58 Info: : 16794: connect.c:2063: Mark not in-flux (was reason 2=connect pid 15061 nwaiters 1 ds /mydata/ttdb)
2011-03-30 17:16:35.58 Info: : 16794: Mark NOT in flux for connect by 15061
2011-03-30 17:16:35.67 Info: : 16794: 16794: <wait not in flux> Finished waiting for in-flux to clear (now reason 0=not in flux pid 15061 nwaiters 0 ds /mydata/ttdb).


(2) We then see subdaemon 16797 produce these messages. It's mistakenly cleaning up objects for pid 15061:

2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: calling ttsClearMutexSessionState 0x7fdc141c38
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: calling kglMutexCleanup 0x7fdc025450
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: return
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttksurus: cag (nil) seg 0x7fdc025368
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: KSS: kssdch: so 0x7fdc025368 = session (0), child count = 0, pass = 0
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: kgttulcuRecoveryByPid: Recovering zombie process 15061
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: kgttulcuRecoveryByPid: Found process object 0x7fdb1368b0 for pid 15061
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: calling ttsClearMutexSessionState 0x7fdc141c38
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: calling kglMutexCleanup 0x7fdc025450
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttsCleanSessionState: return
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: ttksurus: cag (nil) seg 0x7fdc025368
2011-03-30 17:17:00.29 Info: : 16794: 16797/0x2aaaaae13fe0: KSS: kssdch: so 0x7fdc025368 = session (0), child count = 0, pass = 0


(3) pid 15061 then says its disconnecting here:

2011-03-30 17:17:18.95 Info: : 16794: 15061/0x5c16f0: Disconnect /mydata/ttdb
...
2011-03-30 17:17:18.97 Info: : 16794: 15061 0x5c16f0: DisconnectComplete Y /mydata/ttdb
2011-03-30 17:17:18.97 Info: : 16794: daDbDisconnectComplete by 15061: decrementing nUsers from 34, panicked=-1, trashed=-1, shmSeq=3
2011-03-30 17:17:18.97 Info: : 16794: disco.c:616: Mark not in-flux (was reason 3=disconnect pid 15061 nwaiters 0 ds /mydata/ttdb)
2011-03-30 17:17:18.98 Info: : 16794: Starting daRecovery for 15061
2011-03-30 17:17:18.98 Info: : 16794: 15061 ------------------: process exited
2011-03-30 17:17:18.98 Info: : 16794: Finished daRecovery for pid 15061.


(4) We next see PID 19206, a ttIsql session connect:

2011-03-30 17:17:25.52 Info: : 16794: Got hello from pid 19206, type library (/syshome1/app/timesten/TimesTen/11.2.1_ttdb/bin/ttIsqlCmd -v 1 MYCACHE )


(5) PID 19206 then starts chucking out this message. The value 7fdb1368b0 is the "process object" referenced above.

2011-03-30 17:17:25.73 Info: : 16794: 19206/0x60a330: Process 7fdb1368b0 still exists, marked zombie
...100,000 identical lines later...
2011-03-30 17:17:43.82 Info: : 16794: 19206/0x60a330: Process 7fdb1368b0 still exists, marked zombie

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms