smsMaster Dropping Connection to Clients, Showing Error "Lost HB late by x ms" (Doc ID 1206951.1)

Last updated on FEBRUARY 23, 2016

Applies to:

Oracle Communications Network Charging and Control - Version: 3.1.0 and later   [Release: 3.1.0 and later ]
Information in this document applies to any platform.

Symptoms

The main symptom of this issue is clients being disconnected from smsMaster (on the SMS or System Management Server), including clients on the same machine itself.

On SLC and VWS nodes, there will be errors in the syslog (/var/adm/messages) from the various connecting processes.

Clients include:

  • updateLoader
  • smsStatsDaemon
  • smsAlarmDaemon
  • smsMergeDaemon

Relevant errors in the syslog include:

smsMaster Process:

Jun 4 01:37:58 sms1 smsMaster: [ID 675952 user.error] smsMaster(20928) ERROR: Connection pipe broke
Jun 4 01:37:58 sms1 smsMaster: [ID 675952 user.error] smsMaster(20928) ERROR: Write error - Broken pipe

Followed shortly by a reconnection:

Jun 4 01:37:58 sms1 smsMaster: [ID 839465 user.notice] smsMaster(20928) NOTICE: New Connection from x.x.x.x:49449 accepted (FD 9)


Client Processes:

Jun 4 01:39:58 scp1 smsStatsDaemon: [ID 675952 user.error] smsStatsDaemon(1098) ERROR: Connection to 'x.x.x.x' lost HB late by 19968ms


Note: Where x.x.x.x = IP address

Changes

Possible Causes include

  • Recent System upgrade
  • Recent Introduction of smsAlarmManager on the platform
  • Gradually increasing load on the network

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms