Incorrect EXTRA_JAVA_PROPERTIES setting, causes multiple issues while starting AdminServer and WLS_FORMS Managed Server (Doc ID 2271038.1)

Last updated on JUNE 01, 2017

Applies to:

Oracle WebLogic Server - Version 12.2.1.2.0 and later
Information in this document applies to any platform.

Symptoms

Multiple issues will be observed in this case.

1st: (AdminServer)

While AdminServer is starting, below errors will be seen in the startup logs but AdminServer will come up and you will be able to login to Console.

at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:640)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:406)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:346)
2017-05-26 12:39:15.368/31.287 Oracle Coherence GE 12.2.1.2.0 <Info> (thread=[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): Created cache factory com.tangosol.net.DefaultConfigurableCacheFactory
2017-05-26 12:39:15.560/31.479 Oracle Coherence GE 12.2.1.2.0 <Warning> (thread=[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): The cluster name has not been configured, a value of "oracle's cluster" has been automatically generated
2017-05-26 12:39:15.778/31.697 Oracle Coherence GE 12.2.1.2.0 <Info> (thread=[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): Created cache factory com.tangosol.net.DefaultConfigurableCacheFactory
2017-05-26 12:39:16.508/32.428 Oracle Coherence GE 12.2.1.2.0 <Info> (thread=[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)', member=n/a): Created cache factory com.tangosol.net.DefaultConfigurableCacheFactory
<May 26, 2017, 12:39:19,376 PM CDT> <Error> <HTTP> <BEA-101165> <Could not load user defined filter in web.xml: oracle.sysman.eml.app.EMTargetAuthFilter.
java.lang.NullPointerException
at oracle.sysman.eml.app.EMTargetAuthFilter.init(EMTargetAuthFilter.java:120)
at weblogic.servlet.internal.FilterManager$FilterInitAction.run(FilterManager.java:400)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:326)
at weblogic.security.service.SecurityManager.runAsForUserCode(SecurityManager.java:197)
at weblogic.servlet.provider.WlsSecurityProvider.runAsForUserCode(WlsSecurityProvider.java:203)
Truncated. see log file for complete stacktrace
>
<May 26, 2017, 12:39:20,436 PM CDT> <Error> <HTTP> <BEA-101216> <Servlet: "console" failed to preload on startup in Web application: "/em".
javax.servlet.ServletException: Environment variable ORACLE_HOME is null.
at oracle.sysman.eml.app.Console.init(Console.java:434)
at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:343)
at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:294)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:326)
at weblogic.security.service.SecurityManager.runAsForUserCode(SecurityManager.java:197)
Truncated. see log file for complete stacktrace
>
<May 26, 2017, 12:39:20,512 PM CDT> <Error> <Deployer> <BEA-149231> <Unable to set the activation state to true for the application "em".
weblogic.application.ModuleException: javax.servlet.ServletException: Environment variable ORACLE_HOME is null.
at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:140)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:233)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:228)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:45)
Truncated. see log file for complete stacktrace
Caused By: javax.servlet.ServletException: Environment variable ORACLE_HOME is null.
at oracle.sysman.eml.app.Console.init(Console.java:434)
at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:343)
at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:294)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:326)
at weblogic.security.service.SecurityManager.runAsForUserCode(SecurityManager.java:197)
Truncated. see log file for complete stacktrace

2nd (WLS_FORMS)

When managed server was coming up, it was unable to reach the AdminServer which was up and failed to come up in MSI mode with below errors:

<May 24, 2017 11:19:29 AM CDT> <Info> <Management> <BEA-141107> <Version: WebLogic Server 12.2.1.2.0 Mon Oct 3 04:35:36 PDT 2016 1827450>
<May 24, 2017 11:19:30 AM CDT> <Alert> <Management> <BEA-141151> <The Administration Server could not be reached at http://aaa.bbb.ccc.ddd:7001.>
<May 24, 2017 11:19:30 AM CDT> <Info> <Configuration Management> <BEA-150018> <This server is being started in Managed Server independence mode in the absence of the Administration Server.>
<May 24, 2017 11:19:30 AM CDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
<May 24, 2017 11:19:30 AM CDT> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool.>
<May 24, 2017 11:19:30 AM CDT> <Info> <WorkManager> <BEA-002942> <CMM memory level becomes 0. Setting standby thread pool size to 256.>

.......

<May 24, 2017, 11:19:35,360 AM CDT> <Notice> <Security> <BEA-090946> <Security pre-initializing using security realm: myrealm>
<May 24, 2017, 11:19:35,720 AM CDT> <Notice> <Security> <BEA-090947> <Security post-initializing using security realm: myrealm>
<May 24, 2017, 11:19:36,236 AM CDT> <Notice> <Security> <BEA-090082> <Security initialized using administrative security realm: myrealm>
<May 24, 2017, 11:19:36,252 AM CDT> <Critical> <Security> <BEA-090402> <Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.>
<May 24, 2017, 11:19:36,276 AM CDT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: A MultiException has 4 exceptions. They are:
1. weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.security.SecurityService
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.nodemanager.adminserver.NodeManagerMonitorService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.nodemanager.adminserver.NodeManagerMonitorService

A MultiException has 4 exceptions. They are:
1. weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.security.SecurityService
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.nodemanager.adminserver.NodeManagerMonitorService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.nodemanager.adminserver.NodeManagerMonitorService

at org.jvnet.hk2.internal.Collector.throwIfErrors(Collector.java:89)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:249)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:357)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:471)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:232)
Truncated. see log file for complete stacktrace
Caused By: weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:1101)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.postInitialize(CommonSecurityServiceManagerDelegateImpl.java:1220)
at weblogic.security.service.SecurityServiceManager.postInitialize(SecurityServiceManager.java:586)
at weblogic.security.SecurityService.start(SecurityService.java:130)
at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:76)
Truncated. see log file for complete stacktrace
Caused By: javax.security.auth.login.FailedLoginException: [Security:090938]Authentication failure: The specified user failed to log in. weblogic.security.providers.authentication.LDAPAtnDelegateException: [Security:090295]caught unexpected exception, No such object
at com.bea.common.security.utils.ExceptionHandler.throwFailedLoginException(ExceptionHandler.java:62)
at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:368)
at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:117)
at java.security.AccessController.doPrivileged(Native Method)
at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:114)
Truncated. see log file for complete stacktrace
>
<May 24, 2017, 11:19:36,286 AM CDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED.>
<May 24, 2017, 11:19:36,286 AM CDT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.>
<May 24, 2017, 11:19:36,288 AM CDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.>

 

Changes

The setDomainEnv.sh was changed earlier by adding the below lines which caused the issue:

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms