Spark HBase Application on Secure BDA 4.6 Cluster Fails with Error: "ERROR executor.Executor: Exception in task ...InvalidToken: Token has expired ..."
Last updated on NOVEMBER 10, 2017
Applies to:Big Data Appliance Integrated Software - Version 4.6.0 and later
When using a secured (Kerberos) BDA 4.6 cluster, a certain spark streaming application is run daily. The application runs fine for some time and then starts failing. This application reads and writes data from/to HBase by using the cloudera spark-hbase connector (i.e. HBaseContext). To start the application, the principal is passed via kinit to the spark-submit script (via the --principal and --keytab options when running spark-submit). After seven days, when the ticket in the executor cache expires, the application crashes with the following error (as seen in the executors log):
Sign In with your My Oracle Support account
Don't have a My Oracle Support account? Click to get started
Million Knowledge Articles and hundreds of Community platforms