My Oracle Support Banner

Zeppelin Notebooks in Big Data Manager (BDM) Fail with Py4JJavaError: "... token (token for bigdatamgr: HDFS_DELEGATION_TOKEN... is expired" (Doc ID 2460235.1)

Last updated on NOVEMBER 09, 2022

Applies to:

Big Data Cloud Service - Version 1.0 and later
Information in this document applies to any platform.

Symptoms

NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner. 

Running Zeppelin notebooks in Big Data Manager (BDM) version 18.3.1.1302 on a Big Data Cloud Service (BDCS) 4.11.5 cluster, raises the error below:

Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage <STAGE> failed 4 times, most recent failure: Lost task <TASK> in stage <STAGE> (TID <TID>, <bdacloudcluster.domain.oraclecloud.com>, executor <EXECUTOR_NUM>): org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (token for bigdatamgr: HDFS_DELEGATION_TOKEN owner=bigdatamgr@BDACLOUDSERVICE.ORACLE.COM, renewer=yarn, realUser=, issueDate=1536939501155, maxDate=1537544301155, sequenceNumber=<SEQ_NUMBER>, masterKeyId=<MASTER_KEYID>) is expired, current time: 2018-09-21 15:55:18,461+0000 expected renewal time: 2018-09-21 15:38:21,155+0000

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (token for bigdatamgr: HDFS_DELEGATION_TOKEN owner=bigdatamgr@BDACLOUDSERVICE.ORACLE.COM, renewer=yarn, realUser=, issueDate=1536939501155, maxDate=1537544301155, sequenceNumber=<SEQ_NUMBER>, masterKeyId=<MASTER_KEYID>) is expired, current time: 2018-09-21 15:55:18,461+0000 expected renewal time: 2018-09-21 15:38:21,155+0000

This error is happening for all users and these notebooks and scripts previously executed without errors.  There are no known changes in the environment.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.