How to Resolve the Conflict of Port Usage Between Spark and Cloudera Manager (CM) Agent on Edge Node (Doc ID 2034473.1)

Last updated on OCTOBER 11, 2016

Applies to:

Big Data Appliance Integrated Software - Version 3.1.0 and later
Linux x86-64

Goal

On an Edge node executing spark-shell command fails with errors:

spark-shell --master yarn --deploy-mode client
-- > 15/06/25 05:03:58 WARN AbstractLifeCycle: FAILED SocketConnector@0.0.0.0:0: java.net.BindException: Address already in use


And same command works fine when cloudera-scm-agent service on edge node is stopped.

So how to resolve the conflict of port usage between spark and Cloudera Manager (CM) Agent?

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms