Excessive Memory Consumption Caused By Oracle API Gateway CRL Processing (Doc ID 2167559.1)

Last updated on OCTOBER 07, 2022

Applies to:

Symptoms

A production OAG instance is growing in memory consumption until it fails, causing a service outage until it is restarted.  Based on looking at 10 day usage metrics on the OAG Dashboard, it appears to be growing in memory consumption at a rate of about 1gb per 250,000 requests.

It was found that the CRL check task was adding a large amount of processing overhead and causing a significant per-request increase in memory usage by OAG.

The CRL step in question takes a dynamic file URL for the source CRL and is configured to never reload.  Monitoring of LSOF while testing typically showed 100s of file handles open to both the source and OAG CRL cache folder copy of these CRLs.  It appears that OAG is copying the CRL and performing disk IO on every request, despite the CRL already residing in the in-memory cache.  This is inefficient and significantly, degrading production performance, and eventually crashes the system.

The version being used in this case is 11.1.2.3.0

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.