My Oracle Support Banner

ADLS thriuws error while reading parque files - Required field was not present! (Doc ID 2968333.1)

Last updated on JANUARY 17, 2024

Applies to:

Oracle GoldenGate Big Data and Application Adapters - Version 21.3.0.0.0 and later
Information in this document applies to any platform.
Issue is that ADLS occasionally fails while reading the parquet files.

FileReadException: Error while reading file /OPERA.NAME_ADDRESS_CT/.NAME_ADDRESS_CT/20230728-033517584.parquet.
Caused by: IOException: can not read class org.apache.parquet.format.FileMetaData: Required field 'name' was not present! Struct: SchemaElement(type:INT96, repetition_type:REQUIRED, name:null)
Caused by: TProtocolException: Required field 'name' was not present! Struct: SchemaElement(type:INT96, repetition_type:REQUIRED, name:null)
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 61.0 failed 4 times, most recent failure: Lost task 0.3 in stage 61.0 (TID 589) (10.246.34.16 executor 0): com.databricks.sql.io.FileReadException: Error while reading file /OPERA.NAME_ADDRESS_CT/.NAME_ADDRESS_CT/20230728-033517584.parquet.

Goal

 Suspected parquet file corruption

Solution

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Goal
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.