Latency and TimeoutException Seen when a Node is Down on a Oracle Big Data Appliance NoSQL Cluster (Doc ID 1982413.1)

Last updated on OCTOBER 11, 2016

Applies to:

Oracle NoSQL Database - Version 12.1.3.0.5 and later
Linux x86-64

Symptoms

The symptoms are observed on an Oracle Big Data Appliance (BDA) testing NoSQL resiliency on 3/6 node cluster by bringing either the Master or a Storage Node down.


When a NoSQL node either Master/Admin/Storage Node is brought down(using 'shutdown -h now') noticing below issues.

Latency noticed with runadmin call when connecting to a store or nsdbservice

# time java -jar /opt/oracle/kv-ee/lib/kvstore.jar runadmin -host <localhost/AdminNode> -port 5000 -store <StoreName> exit

OR

# time service nsdbservice status

Executing 'aggregate kv -count -key /user'  throws a timeout error on the first call or sometimes the second call

# java -jar $KVHOME/lib/kvstore.jar runadmin -host <localhost/AdminNode> -port 5000 -store <StoreName>
kv-> aggregate kv -count -key /user
Error handling command aggregate kv -count -key /user: oracle.kv.RequestTimeoutException: Request dispatcher: c-2038569192864094728, dispatch timed out after 0 retries. Target: rg4-rn3 (12.1.3.0.14) Timeout: 5000ms
Fault class name: oracle.kv.RequestTimeoutException
kv-> aggregate kv -count -key /user
Row count: 50000



Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms