My Oracle Support Banner

After Applying CPU, Java Patches, WLS Admin Server Hangs and Fails to Start Due to Long Running Thread and/or Log Error: Caused by: oracle.net.ns.NetException: could not resolve the connect identifier (Doc ID 2874212.1)

Last updated on NOVEMBER 02, 2023

Applies to:

Oracle Platform Security for Java - Version 12.2.1.4.0 and later
Information in this document applies to any platform.

Symptoms

After patches are applied (e.g., CPU, java, etc), the WebLogic Server (WLS) Admin Server hangs and does not start.

AdminServer.log shows:

####<May 10, 2022 1:29:29,612 PM EDT> <Info> <WorkManager> <hostname> <AdminServer> <Timer-2> <> <> <ecid> <...> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 4 running threads, 1 idle threads, and 3 standby threads>
####<May 10, 2022 1:31:29,634 PM EDT> <Info> <WorkManager> <hostname> <AdminServer> <Timer-2> <> <> <ecid> <...> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 4 running threads, 1 idle threads, and 3 standby threads>
####<May 10, 2022 1:33:29,648 PM EDT> <Info> <WorkManager> <hostname> <AdminServer> <Timer-2> <> <> <ecid> <...> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 4 running threads, 1 idle threads, and 3 standby threads>
####<May 10, 2022 1:35:29,663 PM EDT> <Info> <WorkManager> <hostname> <AdminServer> <Timer-2> <> <> <ecid> <...> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 4 running threads, 1 idle threads, and 3 standby threads>
####<May 10, 2022 1:37:29,668 PM EDT> <Info> <WorkManager> <hostname> <AdminServer> <Timer-2> <> <> <ecid> <...> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 4 running threads, 1 idle threads, and 3 standby threads>

Reviewing the thread dumps, the below thread is taking long time to complete the operation causing the AdminServer to hang:

[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" #29 daemon prio=5 os_prio=64 tid=0x00000001037e3000 nid=0x22 runnable [0xffffffff79039000]

java.lang.Thread.State: RUNNABLE
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:459)
at sun.nio.ch.Net.connect(Net.java:451)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:646)
- locked <0x00000007a323c1e0> (a java.lang.Object)
- locked <0x00000007a323c200> (a java.lang.Object)
- locked <0x00000007a323c1f0> (a java.lang.Object)
at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
at oracle.net.nt.TimeoutSocketChannel.connect(TimeoutSocketChannel.java:100)
at oracle.net.nt.TimeoutSocketChannel.(TimeoutSocketChannel.java:78)
at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:205)
at oracle.net.nt.ConnOption.connect(ConnOption.java:177)
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:513)
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:555)
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:709)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:322)
at oracle.jdbc.driver.T4CConnection.conanect(T4CConnection.java:1597)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:589)
at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:924)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:58)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:760)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:575)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at oracle.security.jps.internal.policystore.util.JpsDataManagerUtil.getDBConnection(JpsDataManagerUtil.java:405)
at oracle.security.jps.internal.policystore.util.JpsDataManagerUtil.isNewSchema(JpsDataManagerUtil.java:246)
at oracle.security.jps.internal.core.datamanager.DataManagerFactoryImpl.isNewSchema(DataManagerFactoryImpl.java:252)
at oracle.security.jps.internal.core.datamanager.DataManagerFactoryImpl.getDBDataManager(DataManagerFactoryImpl.java:320)
at oracle.security.jps.internal.core.datamanager.DataManagerFactoryImpl.getDataManager(DataManagerFactoryImpl.java:186)
at oracle.security.jps.internal.core.datamanager.DataManagerFactoryImpl.getDataManager(DataManagerFactoryImpl.java:137)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.createDataManager(AbstractSecurityStore.java:96)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersionInternal(AbstractSecurityStore.java:164)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersion(AbstractSecurityStore.java:146)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.init(LdapCredentialStore.java:186)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.start(LdapCredentialStore.java:1021)
at oracle.security.opss.internal.runtime.ServiceContextImpl.start(ServiceContextImpl.java:220)
at oracle.security.opss.internal.runtime.ServiceContextManagerImpl.createDefaultContext(ServiceContextManagerImpl.java:325)
at oracle.security.opss.internal.runtime.ServiceContextManagerImpl.initialize(ServiceContextManagerImpl.java:242)
- locked <0x00000007a2978900> (a oracle.security.opss.internal.runtime.ServiceContextManagerImpl)
at oracle.security.jps.internal.config.OpssCommonStartup$1.run(OpssCommonStartup.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.internal.config.OpssCommonStartup.start(OpssCommonStartup.java:160)
at oracle.security.jps.wls.JpsWlsStartup.start(JpsWlsStartup.java:105)
at oracle.security.jps.JpsStartup.startWithRetry(JpsStartup.java:267)
at oracle.security.jps.JpsStartup.start(JpsStartup.java:232)
- locked <0x00000007407ba670> (a java.lang.Class for oracle.security.jps.JpsStartup)
at oracle.security.jps.wls.JpsDefaultService.start(JpsDefaultService.java:103)
at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:76)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1287
at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:333)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:375)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:487)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:305)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:85)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2126)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:116)
- locked <0x0000000743d715d8> (a java.lang.Object)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:90)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1237)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1168)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:681)
at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)
at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)
at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)
at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)
at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:655)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)
Locked ownable synchronizers:
None

 

Checked:   AdminServer Starts And Gets Hung Without Coming To RUNNING or SHUTDOWN State (Doc ID 2521633.1)

However access the Database (DB) works fine from this same Admin Server host, via sqlplus and via jdbc test connections.

Also another application on the same host but a different home, with same version and patches, works fine using the same database and same jdbc url in all the applicable xml files.

Tried restoring from a complete tar of the top directory from prior the patches application but it still fails, and the log error "Caused by" changes to:

...
Caused by: oracle.net.ns.NetException: could not resolve the connect identifier "//<db hostname>:<port>:<sid>"
at oracle.net.resolver.NameResolver.resolveName(NameResolver.java:183)
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:517)
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:709)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:322)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1597)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:589)
...


Checked:  OPSS - Unable To Start AdminServer -java.sql.SQLRecoverableException: Listener refused the connection with the following error,ORA-12514, TNS:listener does not currently know of service requested in connect descriptor (Doc ID 2089344.1)

However all the jps xml files look correct. Tried to check for hidden characters on the xml files and the jdbc url related entries using vi ":set list" command but problems are seen and it looks exactly as in the other application home in the same server and other servers where it all works.

 

Checked:   Trying to Set Up External Password Store with JDBC Thin Fails With oracle.net.ns.NetException: could not resolve the connect identifier (Doc ID 2400231.1)

However there is no tnsnames.ora on these application servers, only on the DB side, as it is not needed for jdbc connections anyway.

 

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.