My Oracle Support Banner

Spark Jobs Throw "Error [Shut down address: akka.tcp://driverPropsFetcher@<node>:41534]" (Doc ID 2199185.1)

Last updated on APRIL 08, 2020

Applies to:

Big Data Appliance Integrated Software - Version 4.4.0 to 4.4.0 [Release 4.4]
Linux x86-64

Symptoms

NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.

On BDA V4.4 / CDH 5.5.1 the following error is raised when running spark shell jobs:

Error [Shut down address: akka.tcp://driverPropsFetcher@<HOSTNAME>:41534] [
akka.remote.ShutDownAssociation: Shut down address: akka.tcp://driverPropsFetcher@<HOSTNAME>.<DOMAIN>:41534
Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system terminated the association because it is shutting down.]
akka.event.Logging$Error$NoCause$
16/09/28 12:41:19 INFO YarnClientSchedulerBackend: Registered executor: AkkaRpcEndpointRef(Actor[akka.tcp://sparkExecutor@<HOSTNAME>.<DOMAIN>:34898/user/Executor#-935920980]) with ID 29
16/09/28 12:41:19 INFO TaskSetManager: Starting task 0.0 in stage 15.0 (TID 2086, <HOSTNAME>.<DOMAIN>, partition 0,NODE_LOCAL, 2181 bytes)

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.