After Upgrade to BDA V4.5/CDH 5.7.0 Clock Offset Alerts Like "The following health tests are bad: clock offset." are Raised. (Doc ID 2226274.1)

Last updated on JANUARY 27, 2017

Applies to:

Big Data Appliance Integrated Software - Version 4.5.0 and later
Linux x86-64

Symptoms

After upgrade to BDA V4.5/CDH 5.7.0 "clock offset" alerts are raised. The alerts look like:

The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Oct 18, 2016 12:51:46 PM
View Details on bdanode0x.example.com
Monitor Startup: false
Role: NameNode (bdanode01)
Role Type: NameNode
Cluster: <cluster-name>
Cluster Display Name: <cluster-name>
Service: hdfs
Service Display Name: hdfs
Service Type: HDFS
Hosts: bdanode01.example.com
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.

Additional symptoms:

1. The "ntpq -pn" stratum, delay, and offset values are concerning. The values seem to indicate that only one of the 3 NTP servers on the system is usable.

a) "ntpq -pn" from the node with "clock offset" alerts looks like:

 

2. Otherwise
a) /etc/ntp.conf looks ok on the cluster.
b) The ntpd daemons are all up. And ntpd daemon restart does not make any difference.
c) There is no clock drift on the cluster.
d) "dcli -C chkconfig --list ntpd" looks fine.

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms