BDA 4.7/CDH 5.9.0 Cloudera Manager Raises "The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad" (Doc ID 2294503.1)

Last updated on AUGUST 12, 2017

Applies to:

Big Data Appliance Integrated Software - Version 4.7.0 and later
Linux x86-64

Symptoms

On BDA V4.7/CDH 5.9.0/Linux 6.8 the Impala health status in Cloudera Manager shows:

"The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 1 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any."

When the warning occurs, the Impala service goes down. Changing the current max_connections FROM 150 to 550 as described in <Document 2241305.1> Cloudera Manager Randomly Generates Alerts Even to the Point of Bringing Hue and Other Services into "Bad" Health, alleviates the problems for a few weeks, but then the errors start again.

The example here was on a 6 node production BDA cluster and the frequency of the errors does not seem to have any correlation with the number of users or jobs on the system.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms