My Oracle Support Banner

Root File System Is Filling Up With HDFS Edits Files (Doc ID 2438203.1)

Last updated on JULY 20, 2024

Applies to:

Big Data Appliance Integrated Software - Version 4.11.0 and later
Information in this document applies to any platform.

Symptoms

NOTE: In the examples that follow, user details, table name, company name, email, hostnames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.

The root filesystem is filling up with edit files under the NameNode and Journal Node current directories::

/opt/hadoop/dfs/nn/current
/opt/hadoop/dfs/jn/current


The files are not deleted from those folders.

There are some edits_inprogress files which were created seven months ago on the three journal nodes and they are not deleted automatically.

Node01

# cd /opt/hadoop/dfs/jn/xxx-ns/current
# ls -la *progress*
-rw-r--r-- 1 hdfs hadoop 1048576 Jan 30 2018 edits_inprogress_0000000006373850126
-rw-r--r-- 1 hdfs hadoop 1048576 Jan 31 2018 edits_inprogress_0000000006374342713
-rw-r--r-- 1 hdfs hadoop 2097152 Aug 10 17:19 edits_inprogress_0000000007675429757

Node02

# cd /opt/hadoop/dfs/jn/xxx-ns/current
# ls -laptr edits_inprogress*
-rw-r--r-- 1 hdfs hadoop 1048576 Jan 30 2018 edits_inprogress_0000000006373858300
-rw-r--r-- 1 hdfs hadoop 1048576 Jan 31 2018 edits_inprogress_0000000006374341178
-rw-r--r-- 1 hdfs hadoop 3145728 Aug 10 17:18 edits_inprogress_0000000007675402245

Node03


# cd /opt/hadoop/dfs/jn/xxx-ns/current

# ls -laptr edits_inprogress*
-rw-r--r-- 1 hdfs hadoop 3254243 Aug 10 17:14 edits_0000000007675338524-0000000007675368971
-rw-r--r-- 1 hdfs hadoop 2097152 Jan 30 2018 edits_inprogress_0000000006373708541
-rw-r--r-- 1 hdfs hadoop 1048576 Jan 31 2018 edits_inprogress_0000000006374342292
-rw-r--r-- 1 hdfs hadoop 4194304 Aug 10 17:16 edits_inprogress_0000000007675368972

 The journal nodes and NameNodes have not been syncing up for the past seven months.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.