My Oracle Support Banner

[HBASE-HIVE] Limitation in MySQL Does Not Permit Large Serde Values (Jira HIVE-9815) on Oracle Big Data Appliance (BDA) (Doc ID 2098150.1)

Last updated on AUGUST 03, 2021

Applies to:

Big Data Appliance Integrated Software - Version 4.2.0 and later
Linux x86-64

Goal

NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.

There is a [HBASE-HIVE] limitation in MySQL such that it does not permit large serde values.  This is Jira HIVE-9815.

From the following query to MySQL:

mysql> DESCRIBE SERDE_PARAMS;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| SERDE_ID | bigint(20) | NO | PRI | NULL | |
| PARAM_KEY | varchar(256) | NO | PRI | NULL | |
| PARAM_VALUE | varchar(4000) | YES | | NULL | |
+-------------+---------------+------+-----+---------+-------+
3 rows in set (0,02 sec)

This table is created in MySQL with support for 4000 chars in PARAM_VALUE. Since HBase is developed with a large table design in mind, the limit is quickly reached.

As per the blogpost: http://gbif.blogspot.it/2014/03/lots-of-columns-with-hive-and-hbase.html, it is possible to ALTER the table, modifying the datatype to something that handles more characters.

This MOS document addresses the topic of support for modifying the datatype from both the Cloudera and Oracle Big Data Appliance (BDA) perspective.


 

Solution

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Goal
Solution
 Does Cloudera certify TEXT type instead of VARCHAR(4000)?
 Will this change have any impact to Hive, Impala, or the function of the BDA?
 Is there any plan to make a change in the future to support longer characters? In CDH 5.4.7 VARCHAR(4000) is still used.
 If the datatype is manually changed to TEXT, will it revert back to VARCHAR(4000) after upgrading?
 What about columns in other tables that have the same limitation?


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.