NameNodes Will Not Start on BDA V4.2 When Enabling AD Kerberos with Centrify
(Doc ID 2025681.1)
Last updated on DECEMBER 04, 2019
Applies to:
Big Data Appliance Integrated Software - Version 4.2.0 and laterLinux x86-64
Symptoms
NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.
On BDA V4.2 when enabling AD Kerberos, the NameNode service will not come up. The NameNode logs, /var/log/hadoop-hdfs/hadoop-cmf-hdfs-NAMENODE-<HOSTNAME>.<DOMAIN>.log.out, show errors like:
2015-06-10 11:06:56,220 WARN org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:hdfs/host.example.com@EXAMPLE.COM (auth:KERBEROS)
cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))
2015-06-10 11:06:56,221 ERROR org.apache.hadoop.hdfs.server.namenode.EditLogInputStream:
caught exception initializing http://host.example.com:8480/getJournal?jid=cluster-ns&segmentTxId=745246&storageInfo=...
java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException:
No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:464)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445)
at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:455)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
...
PriviledgedActionException as:hdfs/host.example.com@EXAMPLE.COM (auth:KERBEROS)
cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))
2015-06-10 11:06:56,221 ERROR org.apache.hadoop.hdfs.server.namenode.EditLogInputStream:
caught exception initializing http://host.example.com:8480/getJournal?jid=cluster-ns&segmentTxId=745246&storageInfo=...
java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException:
No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:464)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445)
at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:455)
at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
...
This indicates that the NameNodes can not connect with the Journal Nodes when they start up, and so can not authenticate.
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Cause |
Solution |