My Oracle Support Banner

One Journal Node is Out Of Sync Which Causes the NameNode to be in Bad Health (Doc ID 2230831.1)

Last updated on NOVEMBER 08, 2022

Applies to:

Big Data Appliance Integrated Software - Version 4.4.0 and later
Linux x86-64

Symptoms

In Cloudera Manage (CM) a NameNode shows bad health and the error message reports that one of the JournalNode (JN) is out of sync:

JournalNodes out of sync: <HOSTNAME3>.<DOMAIN>. JournalNodes in sync: <HOSTNAME2>.<DOMAIN> <HOSTNAME1>.<DOMAIN>.

Other symptoms can be:

1. The edits number on the healthy JournalNode and on the affected JournalNode may differ.

2. The JournalNode log on the affected host may contain WARNINGS like below:

WARN org.apache.hadoop.hdfs.server.namenode.FSImage: Caught exception after scanning through 0 ops from /<PATH>/jn/<CLUSTER_NAME>/current/edits_inprogress_<#> while determining its valid length. Position was <POS>
java.io.IOException: Can't scan a pre-transactional edit log.

This can happen during upgrade as well as anytime when nodes 1-3 are down for a period of time.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Cause
Solution
 Prerequisites
 Overview of the Steps
 Detailed steps:


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.