Impala Query Fails when Attempting to Select from a Table Created in Hive (Doc ID 1674214.1)

Last updated on OCTOBER 11, 2016

Applies to:

Big Data Appliance Integrated Software - Version 2.4.0 to 3.0 [Release 2.4 to 3.0]
Linux x86-64

Symptoms

On BDA 2.4 using Impala Server version: impalad version 1.2.4 RELEASE in Impala shell unable to view records of table created with parquet.compression.

The same select works fine in hive. The table has been created thru hive.

Tried invalidate metadata command on the table but still getting same error.

Executing the query by logging into Impala shell on the BDA nodes. Tried on different BDA nodes but the same error occurs.

Also able to select from the table thru Impala for a different partition.

Here is the output of different queries on the same table from Impala Shell.

> select * from pg_pos.gpos_cp_fads_parquet limit 10;
Query: select * from pg_pos.gpos_cp_fads_parquet limit 10
ERROR: File hdfs://bdauat-ns/protected/pg_pos/data/staging/gpos_cp_fads_parquet/srce_sys_id=3585/sales_date=2013-03-02/000001_0 has invalid file metadata at file offset 107954.
Error = couldn't deserialize thrift msg: TProtocolException: Invalid data
ERROR: Invalid query handle [<nodename>:21000]

 > select count(*) from pg_pos.gpos_cp_fads_parquet where srce_sys_id=3663;
Query: select count(*) from pg_pos.gpos_cp_fads_parquet where srce_sys_id=3663

ERRORS ENCOUNTERED DURING EXECUTION:
Backend 5:File hdfs://bdauat-ns/protected/pg_pos/data/staging/gpos_cp_fads_parquet/srce_sys_id=3663/sales_date=2014-01-11/000003_0 has invalid file metadata at file offset 122704. Error = couldn't deserialize thrift msg: at type:

ERROR: File hdfs://bdauat-ns/protected/pg_pos/data/staging/gpos_cp_fads_parquet/srce_sys_id=3663/sales_date=2014-01-11/000003_0 has invalid file metadata at file offset 122704. Error = couldn't deserialize thrift msg: at type:
ERROR: Invalid query handle

 > select count(*) from pg_pos.gpos_cp_fads_parquet where srce_sys_id=1359;
Query: select count(*) from pg_pos.gpos_cp_fads_parquet where srce_sys_id=1359
+-----------+
| count(*) |
+-----------+
| 347207816 |
+-----------+
Returned 1 row(s) in 0.93s

> select count(*) from pg_pos.gpos_cp_fads_parquet;
Query: select count(*) from pg_pos.gpos_cp_fads_parquet

ERRORS ENCOUNTERED DURING EXECUTION:
Backend 2:File hdfs://bdauat-ns/protected/pg_pos/data/staging/gpos_cp_fads_parquet/srce_sys_id=3663/sales_date=2014-03-22/000001_0 has invalid file metadata at file offset 122958. Error = couldn't deserialize thrift msg: t type:

ERROR: File hdfs://bdauat-ns/protected/pg_pos/data/staging/gpos_cp_fads_parquet/srce_sys_id=3663/sales_date=2014-03-22/000001_0 has invalid file metadata at file offset 122958. Error = couldn't deserialize thrift msg: t type:
ERROR: Invalid query handle



Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms