My Oracle Support Banner

ECE Java Processes Running Outside Kubernetes Cluster Causing Out of Memory Issues (Doc ID 2889489.1)

Last updated on APRIL 23, 2024

Applies to:

Oracle Communications BRM - Elastic Charging Engine - Version 12.0.0.4.0 and later
Information in this document applies to any platform.

Goal

The user observed Elastic Charging Engine (ECE) related Java process are running outside of Kubernetes cluster instead of running inside pods, and these processes are consuming so much memory which is causing out of memory issues. 

Worker nodes are getting unstable due to excessive memory usage of a Java process triggered by some ECE components.
For example, on worker node 109:
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource Requests Limits
  -------- -------- ------
  CPU 13731m (57%) 15320m (64%)
  memory 20515969024 (25%) 23734347008 (29%)
  ephemeral-storage 270Mi (0%) 300Mi (0%)
  hugepages-1Gi 0 (0%) 0 (0%)
  hugepages-2Mi 0 (0%) 0 (0%)


Events:
[cloud-user@<HOSTNAME> ~]$ kubectl top node <HOSTNAME>
NAME             CPU(cores)  CPU%   MEMORY(bytes)   MEMORY%
<HOSTNAME> 404m         1%       63318Mi              83%
 
Kubernetes is only using around a third of the node resources, while the node reports 83% used.
 
When taking a look at the node itself, find the following:

1. Why are these java processes are running outside cluster?
2. Why these java processes consuming too much memory?

Solution

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Goal
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.