[HBASE-HIVE] Limitation in MySQL Does Not Permit Large Serde Values (Jira HIVE-9815) on Oracle Big Data Appliance (BDA) (Doc ID 2098150.1)

Last updated on JANUARY 20, 2016

Applies to:

Big Data Appliance Integrated Software - Version 4.2.0 and later
Linux x86-64

Goal

There is a [HBASE-HIVE] limitation in MySQL such that it does not permit large serde values.  This is Jira HIVE-9815.

From the following query to MySQL:

mysql> DESCRIBE SERDE_PARAMS;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| SERDE_ID | bigint(20) | NO | PRI | NULL | |
| PARAM_KEY | varchar(256) | NO | PRI | NULL | |
| PARAM_VALUE | varchar(4000) | YES | | NULL | |
+-------------+---------------+------+-----+---------+-------+
3 rows in set (0,02 sec)

This table is created in MySQL with support for 4000 chars in PARAM_VALUE. Since HBase is developed with a large table design in mind, the limit is quickly reached.

As per the blogpost: http://gbif.blogspot.it/2014/03/lots-of-columns-with-hive-and-hbase.html, it is possible to ALTER the table, modifying the datatype to something that handles more characters.

This MOS document addresses the topic of support for modifying the datatype from both the Cloudera and Oracle Big Data Appliance (BDA) perspective.


 

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms