[HBASE-HIVE] Limitation in MySQL Does Not Permit Large Serde Values (Jira HIVE-9815) on Oracle Big Data Appliance (BDA)
(Doc ID 2098150.1)
Last updated on AUGUST 03, 2021
Applies to:
Big Data Appliance Integrated Software - Version 4.2.0 and laterLinux x86-64
Goal
There is a [HBASE-HIVE] limitation in MySQL such that it does not permit large serde values. This is Jira HIVE-9815.
From the following query to MySQL:
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| SERDE_ID | bigint(20) | NO | PRI | NULL | |
| PARAM_KEY | varchar(256) | NO | PRI | NULL | |
| PARAM_VALUE | varchar(4000) | YES | | NULL | |
+-------------+---------------+------+-----+---------+-------+
3 rows in set (0,02 sec)
This table is created in MySQL with support for 4000 chars in PARAM_VALUE. Since HBase is developed with a large table design in mind, the limit is quickly reached.
As per the blogpost: http://gbif.blogspot.it/2014/03/lots-of-columns-with-hive-and-hbase.html, it is possible to ALTER the table, modifying the datatype to something that handles more characters.
This MOS document addresses the topic of support for modifying the datatype from both the Cloudera and Oracle Big Data Appliance (BDA) perspective.
Solution
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Goal |
Solution |
Does Cloudera certify TEXT type instead of VARCHAR(4000)? |
Will this change have any impact to Hive, Impala, or the function of the BDA? |
Is there any plan to make a change in the future to support longer characters? In CDH 5.4.7 VARCHAR(4000) is still used. |
If the datatype is manually changed to TEXT, will it revert back to VARCHAR(4000) after upgrading? |
What about columns in other tables that have the same limitation? |