Parse Errors Received on Dates - oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date
(Doc ID 1455481.1)
Last updated on APRIL 06, 2022
Applies to:
Oracle Loader for Hadoop - Version 1.1.0 and laterLinux x86-64
Symptoms
Running a job using Oracle Loader for Hadoop the job completes however there are parse errors occurring on some of the dates. The job finishes successful however not all inserts are made into the Oracle Database. The following examples show the errors that may be seen.
The output of running the job may show the following:
WARN loader.OraLoaderMapper: skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2000/03/31 00:00:00"
AND
INFO mapred.JobClient: Rows skipped by input error
12/29/06 13:06:56 INFO mapred.JobClient: Parse Error=17
12/29/06 13:06:56 INFO mapred.JobClient: Total rows skipped by input error=17
On the Hadoop Cluster you can check the following to find more information about the warnings and number of rows failed for the job Id reported in Oracle Loader output.
For example:
http://<nodename>:50030/jobtracker.jsp
a) Click on the job Id and look for the Rows Skipped. The Job Tracker UI shows the following detail for the job:
| Rows skipped by input error | Parse Error | 17 | 0 | 17 |
| Total rows skipped by input error | 17 | 0 | 17 |
b) To navigate to the map task log follow below steps
i) Click on map
| Kind | % Complete | Num Tasks | Pending | Running | Complete | Killed | Failed/Killed Task Attempts | |
|---|---|---|---|---|---|---|---|---|
| map | 100.00%
|
1 | 0 | 0 | 1 | 0 | 0 / 0 | |
| reduce | 100.00%
|
1 | 0 | 0 | 1 | 0 | 0 / 0 |
ii) Now click on the task associated with the Map
| Task | Complete | Status | Start Time | Finish Time | Errors | Counters | |
|---|---|---|---|---|---|---|---|
| task_201206270622_0121_m_000000 | 100.00%
|
19-Jul-2012 20:14:22 | 19-Jul-2012 20:14:23 (1sec) | 17 |
iii) In the Task Attempts page click on ALL or 'Last 4KB' in 'Task Logs' column
| Task Attempts | Machine | Status | Progress | Start Time | Finish Time | Errors | Task Logs | Counters | Actions | |
|---|---|---|---|---|---|---|---|---|---|---|
| attempt_201206270622_0151_m_000000_0 | /switch1/bda01/myHostName.example.com | SUCCEEDED | 100.00%
|
19-Jul-2012 20:14:22 | 19-Jul-2012 20:14:23 (1sec) | Last 4KB Last 8KB All |
17 |
|
iv) Review the syslog for WARN associated with oracle.hadoop.loader.OraLoaderMapper in page similar to below url ...
http://myHostName.example.com:50060/tasklog?attemptid=attempt_201206270622_0121_m_000000_0&all=true
2012-07-18 14:51:03,242 WARN oracle.hadoop.loader.OraLoaderMapper: skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2004/03/31 01:01:01"
2012-07-18 14:51:03,242 WARN oracle.hadoop.loader.OraLoaderMapper: skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2008/03/31 02:02:02"
2) Manually check for Warning/Error's in the hadoop logs (usually in /var/log/hadoop) directory for the job Id reported in Oracle Loader output
For example in /var/log/hadoop/userlogs/job_201206270622_0121/attempt_201206270622_0121_m_000000_0 you will find a syslog file which contains warnings like the following:
2012-07-18 14:51:03,242 WARN oracle.hadoop.loader.OraLoaderMapper: skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2004/03/31 01:01:01"
2012-07-18 14:51:03,242 WARN oracle.hadoop.loader.OraLoaderMapper: skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2008/03/31 02:02:02"
3) If using Cloudera Manager, follow below steps to find the syslog of a particular job.
Open Cloudera Manager Web UI , which generally runs on port 7180.
For example:
http://<nodename>:7180/cmf/services/status
Here are steps to navigate to the syslog file:
a) Click on Activites tab in Cloudera Manager.
b) Select 'MapReduce Jobs' from the drop down list.
c) Click on the Job Id reported in Oracle Loader output
d) In Tasks Tab click on the attempt for Task type 'Map.'
For example:- attempt_201206270622_0121_m_000000_0
e) In the logs section click on syslog.
f) In the Log Details you can view the warnings for the syslog.
Example: /var/log/hadoop/userlogs/job_201206270622_0121/attempt_201206270622_0121_m_000000_0/syslog
Jul 18, 3:51:03 PM WARN oracle.hadoop.loader.OraLoaderMapper
skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2000/03/31 00:00:00"
Jul 18, 3:51:03 PM WARN oracle.hadoop.loader.OraLoaderMapper
skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2004/03/31 01:01:01"
Jul 18, 3:51:03 PM WARN oracle.hadoop.loader.OraLoaderMapper
skipping record...oracle.hadoop.loader.OraLoaderException: error parsing input data: Unparseable date: "2008/03/31 02:02:02"
...
...
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |