"OALL8 is in an inconsistent state" Following "Stuck" Thread, "Unstuck" Thread, and "No more data to read from socket" (Doc ID 943327.1)

Last updated on AUGUST 16, 2016

Applies to:

JDBC - Version 10.2.0.3 and later
Information in this document applies to any platform.
***Checked for relevance on 11-Sep-2012***

Symptoms

An application runs on a WebLogic application server and connects to a version 10.2.0.3 database via JDBC/Thin 10.2.0.3 ojdbc14.jar. The application uses a connection pool.

Intermittently, for a period of time, a few times a day, repeated executions of one particular query "spin". Instead of returning results in less than one second (the normal situation) the query fails to return results.

At the times the query "spins", WebLogic logs show "stuck" threads. These threads are waiting at java.net.SocketInputStream.socketRead0.

About 10 minutes after the stuck thread warning, a "No more data to read from socket" exception occurs and the thread becomes unstuck. A few seconds later an "OALL8 is in an inconsistent state" exception occurs and the connection is closed.

The series of events and exceptions are shown in the following excerpts from the WebLogic log.

To remove customer specific information from the log excerpts, some names have been replaced with various combinations of "xxx", "yyy", and "zzz". Also some text has been replaced with "<... text omitted ...>".


<May 28, 2009 9:57:32 AM CDT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "682" seconds working on the request "xxx.yyy.ejb.zzz", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:129)
oracle.net.ns.Packet.receive(Packet.java:239)
oracle.net.ns.DataPacket.receive(DataPacket.java:92)
oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:172)
oracle.net.ns.NetInputStream.read(NetInputStream.java:117)
oracle.net.ns.NetInputStream.read(NetInputStream.java:92)
oracle.net.ns.NetInputStream.read(NetInputStream.java:77)
oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1023)
oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:999)
oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:584)
oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:194)
oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:785)
oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:860)
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1186)
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3381)
oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3425)
oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1490)
weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:128)
...
... lines omitted ...
...
May 28, 2009 10:09:15 AM xxx.common.util.zzz
SEVERE: System exception occured. for Failed to get <... text omitted ...> for 2009-05-27 00:00:00.0
xxx.common.exception.yyyExcepton: <... text omitted ...>
at xxx.yyy.zzz(xxx.java:80)
...
... lines omitted ...
...
Caused by: java.sql.SQLRecoverableException: No more data to read from socket
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:101)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:229)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:403)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1042)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:999)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:584)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:194)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:785)
at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:860)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1186)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3381)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3425)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1490)
at weblogic.jdbc.wrapper.PreparedStatement.executeQuery(PreparedStatement.java:128)
at net.sourceforge.orbroker.Statement.executeQuery(Statement.java:416)
...
... lines omitted ...
...
<May 28, 2009 10:09:16 AM CDT> <Info> <WebLogicServer> <BEA-000339> <[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)' has become "unstuck".> 
...
... lines omitted ...
...
<May 28, 2009 10:09:24 AM CDT> <Error> <JDBC> <BEA-001112> <Test "select count(*) from xxx.yyy" set up for pool "zzz" failed with exception: "java.sql.SQLException: OALL8 is in an inconsistent state".> 
<May 28, 2009 10:09:24 AM CDT> <Info> <JDBC> <BEA-001128> <Connection for pool "zzz" closed.> 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms