Null In Nested JSON Kills Pipeline (Doc ID 2784199.1)

Last updated on JUNE 14, 2021

Applies to:

Goal

Customer is hitting a critical issue causing a failure of an OSA pipeline when it consumes nested JSON containing null value, where a substructure is defined. This issue is semi-deterministic as it kills the pipeline only in the case of concurrently processing nested JSON containing mapped attributes (sample 1.) and nested JSON with null value (sample 2. with nestedAttribute4: null). When there is null at non-nested attribute (sample 3. with attribut1: null) data is processed with no issue as null, or when the nested attribute is missing at all (sample 4. without nestedAttribute4) all attributes expected by the defined data shape (OSA project and screenshot attached) are processed as null without no issue as well.

- Sample data -

- Steps to reproduce -
1. Create a pipeline with Kafka Stream containing JSON with nested schema as in sample data 1.
2. Open the pipeline in draft mode (or publish it, it does not matter)
3. Start to generate stream of Full JSON events to the desired Kafka topic with higher frequency (e.g. 0.3s)
4. Generate a single or multiple events containing sample data 2. (nestedAttribute4 null )
5. Pipeline gets killed

Solution

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.