Upgrade Mammoth to V4.1.0 Fails in Step 7 When Upgrading the Sqoop 2 Service with: Failed to Upgrade Sqoop
(Doc ID 1966882.1)
Last updated on AUGUST 03, 2021
Applies to:
Big Data Appliance Integrated Software - Version 3.0.1 and laterLinux x86-64
Symptoms
Upgrading from BDA Mammoth v3.0.1/CDH 5.0.1 to BDA Mammoth v4.1.0/CDH 5.3 fails to upgrade the Sqoop 2 service. The Mammoth upgrade from 3.0.1 to 4.1.0 raises the following error occur on step 7 of Mammoth script:
************************************
Error [10641]: (//bdanode04.example.com//Stage[main]/Hadoop::Startcloudera/Exec[start_cloudera_services]/returns) change from notrun to 0 failed: /opt/oracle/BDAMammoth/bdaconfig/tmp/startcloudera.sh &> /opt/oracle/BDAMammoth/bdaconfig/tmp/startcloudera_1422662582.out returned 1 instead of one of [0]
************************************
The Sqoop 2 service shows bad health in Cloudera Manager (CM) and clicking on Sqoop 2 service shows the Sqoop 2 Server is "down."
Added Sqoop 2 to Hue as per the last step in "How to Configure the Sqoop 2 Service in Cloudera Manager (CM) for Oracle Big Data Appliance 3.1.0/4.X (Doc ID 1922701.1)" however the Sqoop 2 service does not start.
Re-running mammoth -p fails with:
************************************
Error [10641]: (//bdanode04.example.com//Stage[main]/Hadoop::Upgradecdh/Exec[upgrade_cdh]/returns) change from notrun to 0 failed: /opt/oracle/BDAMammoth/bdaconfig/tmp/upgradecdhparcel.sh &> /opt/oracle/BDAMammoth/bdaconfig/tmp/upgradecdhparcel.out returned 1 instead of one of [0]
************************************
From the Sqoop 2 service in CM you can manually run the "Create Sqoop User Directory" menu option but selecting the options of "Upgrade Sqoop" or "Create Sqoop Database" fail.
When trying to perform the following steps the logs below show the errors:
1. CM > Commands > Upgrade Sqoop
Details stderr.log
+ export LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade/catalina_logging.properties
+ LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade/catalina_logging.properties
+ mkdir -p /var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade/temp
+ EXT_LIBS_DIR=/var/lib/sqoop2
+ cp /usr/share/java/mysql-connector-java.jar /var/lib/sqoop2
+ cp /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar /var/lib/sqoop2
+ cp /usr/share/java/oracle-connector-java.jar /var/lib/sqoop2
+ '[' upgrade == upgrade -o upgrade == create-database ']'
+ export 'JAVA_OPTS=-Xms1073741824 -Xmx1073741824 -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ JAVA_OPTS='-Xms1073741824 -Xmx1073741824 -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ exec /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat/bin/tool-wrapper.sh -server org.apache.sqoop.tomcat.TomcatToolRunner upgrade
Process sqoop-server-upgrade (id=738) on host bdanode04.example.com (id=3) exited with 1 and expected 0
From /var/log/sqoop2/sqoop-cmf-sqoop-SQOOP_SERVER-bdanode04.example.com.log.out:
org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)
at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
at org.apache.sqoop.repository.JdbcRepository.createOrUpgradeRepository(JdbcRepository.java:127)
at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:123)
at org.apache.sqoop.tools.tool.UpgradeTool.runToolWithConfiguration(UpgradeTool.java:39)
at org.apache.sqoop.tools.ConfiguredTool.runTool(ConfiguredTool.java:35)
at org.apache.sqoop.tools.ToolRunner.main(ToolRunner.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.sqoop.tomcat.TomcatToolRunner.main(TomcatToolRunner.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Tool.main(Tool.java:225)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true
at java.sql.DriverManager.getConnection(DriverManager.java:596)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:51)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)
... 17 more
2015-02-02 12:04:47,923 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread
From /var/run/cloudera-scm-agent/process/738-sqoop-server-upgrade stdout.log:
Mon Feb 2 12:04:47 EST 2015
JAVA_HOME=/usr/java/jdk1.7.0_72
using 5 as CDH_VERSION
Sqoop tool executor:
Version: 1.99.4-cdh5.3.0
Revision: 75a4ffb64ddc4001a26a04366271b51b262224a1
Compiled on Tue Dec 16 20:14:23 PST 2014 by jenkins
Running tool: class org.apache.sqoop.tools.tool.UpgradeTool
Upgrade has failed, please check Server logs for further details.
Tool class org.apache.sqoop.tools.tool.UpgradeTool has failed.
2. Create Sqoop Database
Program: sqoop/sqoop.sh ["create-database"]
From /var/run/cloudera-scm-agent/process/739-sqoop-create-database
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Cause |
Solution |
References |