Very Slow Hive and Impala Query Performance and Errors in Logs like: "...java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read.ch ..."
(Doc ID 2519800.1)
Last updated on JUNE 29, 2021
Applies to:Big Data Appliance Integrated Software - Version 4.10.0 and later
Information in this document applies to any platform.
A BDA cluster exhibits increased query times and slow performance when running hive and Impala jobs. In this cluster, users typically access both applications via the web UI in Oozie and hue, but slow performance is also seen with the client applications.
For example, some jobs that normally take 5 minutes are taking more than one hour. Other jobs are taking five times longer and hanging the entire cluster which then requires cluster restart to recover.
There are no errors displayed in the web UI, but there are intermittent errors in the Impala daemon (Impalad) logs like:
"W0302 17:00:15.843978 48829 DFSInputStream.java:865] Exception while reading from BP-569733022-<LOCAL_IP>:<BLOCK_ID> of /<USER>/<USERNAME>/<DIR>/<LOGNAME>.parq from DatanodeInfoWithStorage ...java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/<IP>:<PORT> remote=<DATANODE_FQ_HOSTNAME>/<REMOTE_IP>:<PORT>]"
To view full details, sign in with your My Oracle Support account.
Don't have a My Oracle Support account? Click to get started!
In this Document