Spark job in Yarn Cluster Mode Raises: java.io.IOException: Filesystem closed on the BDA (Doc ID 1980886.1)

Last updated on JUNE 16, 2017

Applies to:

Big Data Appliance Integrated Software - Version 3.0.1 and later
Linux x86-64

Symptoms

Running a Spark job in Yarn Cluster Mode Raises: java.io.IOException: Filesystem closed on the BDA.

The job runs successfully in the spark shell, but not through pyspark.  The job is run as:




Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms