My Oracle Support Banner

Change From Standard To Summertime Caused Extended Outage In Prod: JTA Lease Lost - Cluster using Database Leasing (Doc ID 1590774.1)

Last updated on DECEMBER 05, 2016

Applies to:

Oracle Weblogic Server - Version 10.3.5 and later
Information in this document applies to any platform.

Symptoms

The change from standard to summertime happens at 2 AM: the clock is moved one hour forward, i.e. from 2 AM to 3 AM immediately. This caused a large outage where many XA Cluster instances shut itself down with the following message:

<Mar 31, 2013 3:00:09 AM CEST> <Critical> <Health> <BEA-310006> <Critical Subsystem JTAMigratableGroup has failed. Setting server state to FAILED.
Reason: JTA migratable target lost a lease>
<Mar 31, 2013 3:00:09 AM CEST> <Critical> <WebLogicServer> <BEA-000385> <Server health failed. Reason: health of critical service 'JTAMigratableGroup' failed>
<Mar 31, 2013 3:00:09 AM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>

Some issues regarding recovery a FileStore based TLOG could be also noticed in the logs:

<JTA> <BEA-110213> <The activation of Transaction Recovery Service for server [...] failed.
Caused By: java.io.IOException: Error from fcntl() for file locking, Resource temporarily unavailable, errno=11

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.