Spark Jobs Throw "Error [Shut down address: akka.tcp://driverPropsFetcher@<node>:41534]"

(Doc ID 2199185.1)

Last updated on NOVEMBER 20, 2016

Applies to:

Big Data Appliance Integrated Software - Version 4.4.0 to 4.4.0 [Release 4.4]
Linux x86-64


On BDA V4.4 / CDH 5.5.1 the following error is raised when running spark shell jobs:

Error [Shut down address: akka.tcp://driverPropsFetcher@<node>:41534] [
akka.remote.ShutDownAssociation: Shut down address: akka.tcp://driverPropsFetcher@<>:41534
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system terminated the association because it is shutting down.
16/09/28 12:41:19 INFO YarnClientSchedulerBackend: Registered executor: AkkaRpcEndpointRef(Actor[akka.tcp://sparkExecutor@<>:34898/user/Executor#-935920980]) with ID 29
16/09/28 12:41:19 INFO TaskSetManager: Starting task 0.0 in stage 15.0 (TID 2086, <>, partition 0,NODE_LOCAL, 2181 bytes)



Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms