YARN Resource Manager "Reserved Memory" and "Reserved Cores" Are Higher than Allocated Resources Causing Users to be Unable to Schedule Yarn Jobs After BDA 4.9 Upgrade

(Doc ID 2313773.1)

Last updated on OCTOBER 08, 2017

Applies to:

Big Data Appliance Integrated Software - Version 4.9.0 and later
Linux x86-64

Symptoms

After upgrading a 6 node Mammoth cluster from 4.8/CDH 5.10.1 to 4.9/CDH 5.11.1 where each node has 88TB and 256 GB RAM with 88V cores, the Hadoop Yarn Resource Manager shows higher memory and vcores reserved than those that are actually allocated.

For example, Hadoop shows Cluster Metrics with values like:

Due to this problem, users are unable to run any new jobs.

Stopping the services and trying to clear the issue does not resolve it.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms