MultiDatasource containing LLR enabled RAC Datasources causes WebLogic Server instance to fail on startup (Doc ID 1550727.1)

Last updated on NOVEMBER 05, 2016

Applies to:

Oracle Weblogic Server - Version 10.3.3 to 10.3.6
Information in this document applies to any platform.

Symptoms

A WebLogic MultiDataSource (MDS) contains one or more WebLogic RAC DataSources (DSs), and these DSs are Logging Last Resource (LLR) enabled. When this MDS is targetted to a WebLogic instance, this WebLogic instance can fail to startup depending on the situation.

LLR is a type of WebLogic global transaction protocol that can be configured for a datasource that runs with Non-XA JDBC Driver. Other types are emulate-two-phase-commit and one-phase-commit.

 

The error types that can be encountered are:

####<Apr 9, 2013 9:03:28 AM EEST> <Critical> <JTA> <MACHINE> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <4fdef9521b3f9c0b:5f36f916:13ded641670:-8000-0000000000000003> <1365487408385> <BEA-110482> <A logging last resource failed during initialization. The server cannot boot unless all configured logging last resources (LLRs) initialize. Failing reason:
weblogic.transaction.loggingresource.LoggingResourceException: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: The Network Adapter could not establish the connection

weblogic.transaction.loggingresource.LoggingResourceException: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: The Network Adapter could not establish the connection
    at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.recoverXARecords(JTSLoggableResourceImpl.java:248)
    at weblogic.transaction.internal.ServerTransactionManagerImpl$RegisteredLoggingResourceTransactions.registerLoggingResourceTransactions(ServerTransactionManagerImpl.java:801)
    at weblogic.transaction.internal.ServerTransactionManagerImpl.registerLoggingResourceTransactions(ServerTransactionManagerImpl.java:767)
    at weblogic.jdbc.common.internal.RmiDataSource.recoverLoggingResourceTransactions(RmiDataSource.java:400)

...
...
Caused By: weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: The Network Adapter could not establish the connection
    at weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:250)
    at weblogic.jdbc.jts.Driver.newConnection(Driver.java:938)
    at weblogic.jdbc.jts.Driver.newConnection(Driver.java:879)
    at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.getNewJTSConnection(JTSLoggableResourceImpl.java:503)
    at weblogic.jdbc.wrapper.JTSLoggableResourceImpl.recoverXARecords(JTSLoggableResourceImpl.java:237)
    at weblogic.transaction.internal.ServerTransactionManagerImpl$RegisteredLoggingResourceTransactions.registerLoggingResourceTransactions(ServerTransactionManagerImpl.java:801)
...
...





The debug flags that give extensive usefull information in this case are:


The error in this case is:

####<Apr 9, 2013 9:03:28 AM EEST> <Critical> <JTA> <MACHINE> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <8f0524879b7593da:4f38c37d:13e60c51881:-8000-0000000000000003> <1367423138965> <BEA-110482> <A logging last resource failed during initialization. The server cannot boot unless all configured logging last resources (LLRs) initialize. Failing reason:
weblogic.common.ResourceException: All Resources member of the MultiDataSource are either unhealthy or shutdown

weblogic.common.ResourceException: All Resources member of the MultiDataSource are either unhealthy or shutdown
    at weblogic.jdbc.common.internal.MultiPool.checkLLRDataSource(MultiPool.java:432)
    at weblogic.jdbc.common.internal.MultiPool.start(MultiPool.java:218)
    at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:481)
    at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:428)
    at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:280)
    at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:199)
    at weblogic.application.internal.flow.DeploymentCallbackFlow$1.next(DeploymentCallbackFlow.java:517)
    at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
    at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:159)
    at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:45)
    at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:648)
    at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
    at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:191)
    at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:44)
    at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:154)
    at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
    at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:55)
    at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
    at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
    at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
    at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
    at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
    at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
    at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:180)
    at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:96)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

 

Changes

NA

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms