Spark HBase Application on Secure BDA 4.6 Cluster Fails with Error: "ERROR executor.Executor: Exception in task ...InvalidToken: Token has expired ..."

(Doc ID 2324273.1)

Last updated on NOVEMBER 10, 2017

Applies to:

Big Data Appliance Integrated Software - Version 4.6.0 and later
Linux x86-64

Symptoms

When using a secured (Kerberos) BDA 4.6 cluster, a certain spark streaming application is run daily.  The application runs fine for some time and then starts failing. This application reads and writes data from/to HBase by using the cloudera spark-hbase connector (i.e. HBaseContext). To start the application, the principal is passed via kinit to the spark-submit script (via the --principal and --keytab options when running spark-submit).  After seven days, when the ticket in the executor cache expires, the application crashes with the following error (as seen in the executors log):

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms