After Installing Keytrustee 5.3.0 Jobs Raise "java.net.ConnectException: Connection refused"
(Doc ID 2091808.1)
Last updated on AUGUST 03, 2021
Applies to:
Big Data Appliance Integrated Software - Version 4.2.0 and laterLinux x86-64
Symptoms
All jobs run as any user fail after installing/configuring Keytrustee 5.3.0 on a BDA V4.2/CDH 5.4.4 cluster as below:
[oracle@<HOSTNAME1> ~]$ hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.4.4.jar pi 10 1000
Number of Maps = 10
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
15/12/21 14:17:49 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 2910 for oracle on ha-hdfs:<CLUSTER_NAME>-ns
15/12/21 14:17:50 WARN security.UserGroupInformation: PriviledgedActionException as:oracle@<REALM> (auth:KERBEROS) cause:java.net.ConnectException: Connection refused
15/12/21 14:17:50 WARN security.UserGroupInformation: PriviledgedActionException as:oracle@<REALM> (auth:KERBEROS) cause:java.io.IOException: java.net.ConnectException: Connection refused
java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:887)
at org.apache.hadoop.crypto.key.KeyProviderDelegationTokenExtension.addDelegationTokens(KeyProviderDelegationTokenExtension.java:86)
at org.apache.hadoop.hdfs.DistributedFileSystem.addDelegationTokens(DistributedFileSystem.java:2056)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:140)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1324)
at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketI
Number of Maps = 10
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
15/12/21 14:17:49 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 2910 for oracle on ha-hdfs:<CLUSTER_NAME>-ns
15/12/21 14:17:50 WARN security.UserGroupInformation: PriviledgedActionException as:oracle@<REALM> (auth:KERBEROS) cause:java.net.ConnectException: Connection refused
15/12/21 14:17:50 WARN security.UserGroupInformation: PriviledgedActionException as:oracle@<REALM> (auth:KERBEROS) cause:java.io.IOException: java.net.ConnectException: Connection refused
java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:887)
at org.apache.hadoop.crypto.key.KeyProviderDelegationTokenExtension.addDelegationTokens(KeyProviderDelegationTokenExtension.java:86)
at org.apache.hadoop.hdfs.DistributedFileSystem.addDelegationTokens(DistributedFileSystem.java:2056)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:140)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1324)
at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketI
The kms-catalina.2015-12-18.log complains about permission denied on keystore.jks. For example:
java.io.FileNotFoundException: /opt/cloudera/security/jks/keystore.jks (Permission denied)
The commands 'hadoop key list' and 'hadoop create key' do not work. The command 'hadoop create key' throws an error in the kms.log saying the password is tampered with or incorrect for keystore:
2015-12-21 18:48:49,866 DEBUG com.cloudera.keytrustee.TrusteeKeyProvider: getKeyVersions threw KeyTrusteeException
com.cloudera.keytrustee.KeyTrusteeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:949)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDeposits(ClientConnectionImpl.java:899)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.cloudera.keytrustee.ClientConnectionInvocationHandler.invoke(ClientConnectionInvocationHandler.java:46)
at com.sun.proxy.$Proxy5.getDeposits(Unknown Source)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:982)
at com.cloudera.keytrustee.TrusteeKeyProvider.getHandleDeposits(TrusteeKeyProvider.java:927)
at com.cloudera.keytrustee.TrusteeKeyProvider.getKeyVersions(TrusteeKeyProvider.java:400)
at com.cloudera.keytrustee.TrusteeKeyProvider.keyExists(TrusteeKeyProvider.java:682)
at com.cloudera.keytrustee.TrusteeKeyProvider.createKey(TrusteeKeyProvider.java:307)
at org.apache.hadoop.crypto.key.KeyProvider.createKey(KeyProvider.java:515)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.createKey(KeyAuthorizationKeyProvider.java:160)
at org.apache.hadoop.crypto.key.kms.server.KMS$1.run(KMS.java:135)
at org.apache.hadoop.crypto.key.kms.server.KMS$1.run(KMS.java:132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.crypto.key.kms.server.KMS.createKey(KMS.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.hadoop.crypto.key.kms.server.KMSMDCFilter.doFilter(KMSMDCFilter.java:84)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:277)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:555)
at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at javax.net.ssl.DefaultSSLSocketFactory.throwException(SSLSocketFactory.java:248)
at javax.net.ssl.DefaultSSLSocketFactory.createSocket(SSLSocketFactory.java:255)
at sun.net.www.protocol.https.HttpsClient.createSocket(HttpsClient.java:405)
at sun.net.NetworkClient.doConnect(NetworkClient.java:162)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1104)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:998)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1282)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1257)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:163)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:120)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:114)
at com.cloudera.keytrustee.impl.EncryptedPayloadConnection.fetchData(EncryptedPayloadConnection.java:78)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:945)
... 64 more
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at java.security.Provider$Service.newInstance(Provider.java:1617)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
at javax.net.ssl.SSLSocketFactory.getDefault(SSLSocketFactory.java:122)
at javax.net.ssl.HttpsURLConnection.getDefaultSSLSocketFactory(HttpsURLConnection.java:332)
at javax.net.ssl.HttpsURLConnection.<init>(HttpsURLConnection.java:289)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.<init>(HttpsURLConnectionImpl.java:85)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:62)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:57)
at java.net.URL.openConnection(URL.java:972)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:131)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:120)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:114)
at com.cloudera.keytrustee.impl.EncryptedPayloadConnection.fetchData(EncryptedPayloadConnection.java:78)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:945)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDeposits(ClientConnectionImpl.java:899)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.cloudera.keytrustee.ClientConnectionInvocationHandler.invoke(ClientConnectionInvocationHandler.java:46)
at com.sun.proxy.$Proxy5.getDeposits(Unknown Source)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:982)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:905)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:897)
at com.cloudera.keytrustee.TrusteeKeyProvider.getKeys(TrusteeKeyProvider.java:519)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.getKeys(KeyAuthorizationKeyProvider.java:283)
at org.apache.hadoop.crypto.key.kms.server.KMS$5.run(KMS.java:259)
at org.apache.hadoop.crypto.key.kms.server.KMS$5.run(KMS.java:256)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.crypto.key.kms.server.KMS.getKeyNames(KMS.java:255)
... 41 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:780)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56)
at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:225)
at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70)
at java.security.KeyStore.load(KeyStore.java:1445)
at sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
at sun.security.ssl.SSLContextImpl$DefaultSSLContext.getDefaultTrustManager(SSLContextImpl.java:767)
at sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:733)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.security.Provider$Service.newInstance(Provider.java:1595)
... 78 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:778)
... 90 more
com.cloudera.keytrustee.KeyTrusteeException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:949)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDeposits(ClientConnectionImpl.java:899)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.cloudera.keytrustee.ClientConnectionInvocationHandler.invoke(ClientConnectionInvocationHandler.java:46)
at com.sun.proxy.$Proxy5.getDeposits(Unknown Source)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:982)
at com.cloudera.keytrustee.TrusteeKeyProvider.getHandleDeposits(TrusteeKeyProvider.java:927)
at com.cloudera.keytrustee.TrusteeKeyProvider.getKeyVersions(TrusteeKeyProvider.java:400)
at com.cloudera.keytrustee.TrusteeKeyProvider.keyExists(TrusteeKeyProvider.java:682)
at com.cloudera.keytrustee.TrusteeKeyProvider.createKey(TrusteeKeyProvider.java:307)
at org.apache.hadoop.crypto.key.KeyProvider.createKey(KeyProvider.java:515)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.createKey(KeyAuthorizationKeyProvider.java:160)
at org.apache.hadoop.crypto.key.kms.server.KMS$1.run(KMS.java:135)
at org.apache.hadoop.crypto.key.kms.server.KMS$1.run(KMS.java:132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.crypto.key.kms.server.KMS.createKey(KMS.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.hadoop.crypto.key.kms.server.KMSMDCFilter.doFilter(KMSMDCFilter.java:84)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:277)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:555)
at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at javax.net.ssl.DefaultSSLSocketFactory.throwException(SSLSocketFactory.java:248)
at javax.net.ssl.DefaultSSLSocketFactory.createSocket(SSLSocketFactory.java:255)
at sun.net.www.protocol.https.HttpsClient.createSocket(HttpsClient.java:405)
at sun.net.NetworkClient.doConnect(NetworkClient.java:162)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1104)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:998)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1282)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1257)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:163)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:120)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:114)
at com.cloudera.keytrustee.impl.EncryptedPayloadConnection.fetchData(EncryptedPayloadConnection.java:78)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:945)
... 64 more
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
at java.security.Provider$Service.newInstance(Provider.java:1617)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
at javax.net.ssl.SSLSocketFactory.getDefault(SSLSocketFactory.java:122)
at javax.net.ssl.HttpsURLConnection.getDefaultSSLSocketFactory(HttpsURLConnection.java:332)
at javax.net.ssl.HttpsURLConnection.<init>(HttpsURLConnection.java:289)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.<init>(HttpsURLConnectionImpl.java:85)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:62)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:57)
at java.net.URL.openConnection(URL.java:972)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:131)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:120)
at com.cloudera.keytrustee.URLConnector.execQuery(URLConnector.java:114)
at com.cloudera.keytrustee.impl.EncryptedPayloadConnection.fetchData(EncryptedPayloadConnection.java:78)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDepositsJSON(ClientConnectionImpl.java:945)
at com.cloudera.keytrustee.impl.ClientConnectionImpl.getDeposits(ClientConnectionImpl.java:899)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.cloudera.keytrustee.ClientConnectionInvocationHandler.invoke(ClientConnectionInvocationHandler.java:46)
at com.sun.proxy.$Proxy5.getDeposits(Unknown Source)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:982)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:905)
at com.cloudera.keytrustee.TrusteeKeyProvider.getAllDeposits(TrusteeKeyProvider.java:897)
at com.cloudera.keytrustee.TrusteeKeyProvider.getKeys(TrusteeKeyProvider.java:519)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.KeyProviderExtension.getKeys(KeyProviderExtension.java:90)
at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.getKeys(KeyAuthorizationKeyProvider.java:283)
at org.apache.hadoop.crypto.key.kms.server.KMS$5.run(KMS.java:259)
at org.apache.hadoop.crypto.key.kms.server.KMS$5.run(KMS.java:256)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.crypto.key.kms.server.KMS.getKeyNames(KMS.java:255)
... 41 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:780)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56)
at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:225)
at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70)
at java.security.KeyStore.load(KeyStore.java:1445)
at sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
at sun.security.ssl.SSLContextImpl$DefaultSSLContext.getDefaultTrustManager(SSLContextImpl.java:767)
at sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:733)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.security.Provider$Service.newInstance(Provider.java:1595)
... 78 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:778)
... 90 more
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |