BDA 4.7/CDH 5.9.0 Cloudera Manager Raises "The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad"
Last updated on AUGUST 12, 2017
Applies to:Big Data Appliance Integrated Software - Version 4.7.0 and later
On BDA V4.7/CDH 5.9.0/Linux 6.8 the Impala health status in Cloudera Manager shows:
"The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 1 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any."
When the warning occurs, the Impala service goes down. Changing the current max_connections FROM 150 to 550 as described in <Document 2241305.1> Cloudera Manager Randomly Generates Alerts Even to the Point of Bringing Hue and Other Services into "Bad" Health, alleviates the problems for a few weeks, but then the errors start again.
The example here was on a 6 node production BDA cluster and the frequency of the errors does not seem to have any correlation with the number of users or jobs on the system.
Sign In with your My Oracle Support account
Don't have a My Oracle Support account? Click to get started
Million Knowledge Articles and hundreds of Community platforms