Solr Health Turns Bad after Enabling Kerberos (Doc ID 2042649.1)

Last updated on JUNE 15, 2017

Applies to:

Big Data Appliance Integrated Software - Version 4.2.0 and later
Linux x86-64

Goal


Solr service is on two nodes in the cluster. However checked and found the role log details showing a kerberos initialization error similar to the following:

2015-07-16 14:19:00,061 INFO org.apache.solr.core.HdfsDirectoryFactory: Attempting to acquire kerberos ticket with keytab: solr.keytab, principal: solr/<node>@<REALM>
2015-07-16 14:19:00,066 INFO org.apache.solr.core.SolrCore: [lab2_shard1_replica2] Opening new SolrCore at /var/lib/solr/lab2_shard1_replica2/, dataDir=hdfs://<node>-ns/solr/lab2/core_node1/data/
2015-07-16 14:19:00,067 ERROR org.apache.solr.core.CoreContainer: Error creating core [lab1_shard2_replica1]: java.io.IOException: Login failure for solr/<node>@<REALM> from keytab solr.keytab: javax.security.auth.login.LoginException: Checksum failed
java.lang.RuntimeException: java.io.IOException: Login failure for solr/<node>@<REALM> from keytab solr.keytab: javax.security.auth.login.LoginException: Checksum failed
  at org.apache.solr.core.HdfsDirectoryFactory.initKerberos(HdfsDirectoryFactory.java:373)
  at org.apache.solr.core.HdfsDirectoryFactory.init(HdfsDirectoryFactory.java:104)
  at org.apache.solr.core.SolrCore.initDirectoryFactory(SolrCore.java:467)
  at org.apache.solr.core.SolrCore.(SolrCore.java:701)
  at org.apache.solr.core.SolrCore.(SolrCore.java:657)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Login failure for solr/<node>@<REALM> from keytab solr.keytab: javax.security.auth.login.LoginException: Checksum failed
  at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:976)
  at org.apache.solr.core.HdfsDirectoryFactory.initKerberos(HdfsDirectoryFactory.java:371)
  ... 11 more
Caused by: javax.security.auth.login.LoginException: Checksum failed
  at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:804)
  at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
  at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
  at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
  at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:967)
  ... 12 more
Caused by: KrbException: Checksum failed
  at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:102)
  at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:94)
  at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:175)
  at sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:149)
  at sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
  at sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:285)
 

 

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms