Spark HBase Application on Secure BDA 4.6 Cluster Fails with Error: "ERROR executor.Executor: Exception in task ...InvalidToken: Token has expired ..." (Doc ID 2324273.1)

Last updated on NOVEMBER 08, 2022

Applies to:

Symptoms

When using a secured (Kerberos) BDA 4.6 cluster, a certain spark streaming application is run daily.  The application runs fine for some time and then starts failing. This application reads and writes data from/to HBase by using the cloudera spark-hbase connector (i.e. HBaseContext). To start the application, the principal is passed via kinit to the spark-submit script (via the --principal and --keytab options when running spark-submit).  After seven days, when the ticket in the executor cache expires, the application crashes with the following error (as seen in the executors log):

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.