Unable to Access Portal Managed Server Status Is Failed_not_restartable (Doc ID 1626636.1)

Last updated on JUNE 29, 2017

Applies to:

Portal - Version 11.1.1.6.0 and later
Information in this document applies to any platform.

Symptoms

Trying to access Oracle Portal, get the message:

An error occurred while processing the request, Try refreshing your browser. If problem persists contact the site administrator.

In the WebLogic Server Administration Console, Portal Managed Server status is Failed_not_restartable instead of Running.
The portal.out log file shows the managed server was RUNNING, then some java dumps  occurred and after that the sever fails: 

[...]
<Jan 17, 2014 6:54:11 PM GMT+05:30> <Notice> <Cluster> <BEA-000162> <Starting "async" replication service with remote cluster address "null">
<Jan 17, 2014 6:54:11 PM GMT+05:30> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on aa.bb.cc.dd:9001 for protocols iiop, t3, CLUSTER-BROADCAST, ldap, snmp, http.>
<Jan 17, 2014 6:54:11 PM GMT+05:30> <Notice> <WebLogicServer> <BEA-000330> <Started WebLogic Managed Server "WLS_PORTAL2" for domain "portal_domain" running in Production Mode>
<Jan 17, 2014 6:54:11 PM GMT+05:30> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING>
<Jan 17, 2014 6:54:11 PM GMT+05:30> <Notice> <WebLogicServer> <BEA-000360> <Server started in RUNNING mode>
JVMDUMP039I Processing dump event "abort", detail "" at 2014/01/17 18:54:12 - please wait.
JVMDUMP032I JVM requested System dump using '/u01/apps/oracle/product/fmw/user_projects/domains/portal_domain/core.20140117.185412.675888.0001.dmp' in response to an event
Note: "Enable full CORE dump" in smit is set to FALSE and as a result there will be limited threading information in core file.
JVMDUMP012E Error in System dump: cannot find core file: "No such file or directory". check "ulimit -Hc" is set high enough
JVMDUMP032I JVM requested Java dump using '/u01/apps/oracle/product/fmw/user_projects/domains/portal_domain/javacore.20140117.185412.675888.0002.txt' in response to an event
JVMDUMP010I Java dump written to /u01/apps/oracle/product/fmw/user_projects/domains/portal_domain/javacore.20140117.185412.675888.0002.txt
JVMDUMP032I JVM requested Snap dump using '/u01/apps/oracle/product/fmw/user_projects/domains/portal_domain/Snap.20140117.185412.675888.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /u01/apps/oracle/product/fmw/user_projects/domains/portal_domain/Snap.20140117.185412.675888.0003.trc
JVMDUMP013I Processed dump event "abort", detail "".
<Jan 17, 2014 6:54:16 PM> <FINEST> <NodeManager> <Waiting for the process to die: null>
<Jan 17, 2014 6:54:16 PM> <INFO> <NodeManager> <Server failed but will not be restarted because the maximum number of restart attempts has been exceeded>
<Jan 17, 2014 6:54:16 PM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>


The Admin server log file shows the following information:

[...]
####<Jan 17, 2014 6:50:43 PM GMT+05:30> <Error> <NodeManager> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-000000000000019e> <1389964843760> <BEA-300048> <Unable to start the server WLS_PORTAL1 : Exception while starting server 'WLS_PORTAL1'>
####<Jan 17, 2014 6:50:45 PM GMT+05:30> <Info> <Server> <myserver.oracle.com> <AdminServer> <[STANDBY] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001a0> <1389964845598> <BEA-002634> <The server "WLS_PORTAL" disconnected from this server.>
####<Jan 17, 2014 6:50:45 PM GMT+05:30> <Info> <JMX> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001a1> <1389964845598> <BEA-149507> <JMX Connectivity has been discontinued with the managed server WLS_PORTAL.>
####<Jan 17, 2014 6:50:47 PM GMT+05:30> <Info> <Health> <myserver.oracle.com> <AdminServer> <weblogic.GCMonitor> <<anonymous>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-000000000000000e> <1389964847974> <BEA-310002> <16% of the total memory in the server is free>
####<Jan 17, 2014 6:51:02 PM GMT+05:30> <Info> <Server> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001ab> <1389964862766> <BEA-002635> <The server "WLS_PORTAL2" connected to this server.>
####<Jan 17, 2014 6:51:02 PM GMT+05:30> <Info> <JMX> <myserver.oracle.com> <AdminServer> <[STANDBY] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001ac> <1389964862964> <BEA-149506> <Established JMX Connectivity with WLS_PORTAL2 at the JMX Service URL of service:jmx:t3://myserver.oracle.com:9001/jndi/weblogic.management.mbeanservers.runtime.>
####<Jan 17, 2014 6:51:05 PM GMT+05:30> <Info> <Server> <myserver.oracle.com> <AdminServer> <[STANDBY] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001ad> <1389964865040> <BEA-002634> <The server "WLS_PORTAL2" disconnected from this server.>
####<Jan 17, 2014 6:51:05 PM GMT+05:30> <Info> <JMX> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001ae> <1389964865040> <BEA-149507> <JMX Connectivity has been discontinued with the managed server WLS_PORTAL2.>
####<Jan 17, 2014 6:52:16 PM GMT+05:30> <Info> <Server> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9e2c6d984dab3e1b:-5bd9709:143a0469e27:-8000-00000000000001bd> <1389964936194> <BEA-002635> <The server "WLS_PORTAL" connected to this server.>
[...]
####<Jan 17, 2014 9:13:15 PM GMT+05:30> <Error> <NodeManager> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <9e2c6d984dab3e1b:672b32d6:143a0d6f3dc:-8000-0000000000000031> <1389973395094> <BEA-300033> <Could not execute command "getVersion" on the node manager. Reason: "Connection refused. Could not connect to NodeManager. Check that it is running at myserver.oracle.com:5558.".>
####<Jan 17, 2014 9:13:15 PM GMT+05:30> <Error> <NodeManager> <myserver.oracle.com> <AdminServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <9e2c6d984dab3e1b:672b32d6:143a0d6f3dc:-8000-0000000000000031> <1389973395100> <BEA-300033> <Could not execute command "getVersion" on the node manager. Reason: "Connection refused. Could not connect to NodeManager. Check that it is running at myserver.oracle.com:5558.".>  




Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms