ESB Cluster esb_containerid_lease table is Missing Lease ID for some of the Nodes in the Cluster (Doc ID 1400338.1)

Last updated on FEBRUARY 07, 2012

Applies to:

Oracle ESB - Version: 10.1.3.5 and later   [Release: and later ]
Information in this document applies to any platform.

Symptoms

In a ESB clustered environment, some nodes are missing their lease id in the esb_containerid_lease table. This causes multiple nodes from same cluster  to process the same message. This problem only occurs when esb_group_status table has a large amount of data like 7+ million rows.

Steps to reproduce:

1. Start Node_1
2. Observe node_1 locking  esb_containerid_lease table and start performing updates on esb_group_status table.
3. Start Node_2
4. Node_1 has esb_containerid_lease locked and Node_2 can not update its heartbeat in esb_containerid_lease table
5. When Node_1 is done running the SQL update statements (10-15 minutes later), Node_2 has a timestamp in esb_containerid_lease that is old and appears stale.
6. Node_1 assumes Node_2 is offline since its heartbeat is old and removes it from esb_containerid_lease and also updates esb_group_status to change the lockconid for Node_2's rows.
7. As a result, only one row appears in esb_containerid_lease, but both nodes are still processing messages.


Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms