Communications Outbound Manager Component Crashes when trying to FTP a file (Doc ID 1919424.1)

Last updated on MARCH 28, 2024

Applies to:

Symptoms

CommOutboundMgr component is crashing sporadically in production environment. The following error can be observed in Siebel Enterprise Server log:

ServerLog ComponentUpdate 2 0000245b53e06913:0 2014-08-05 03:39:43 CommOutboundMgr STARTING Component is starting up.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 03:39:44 Created multithreaded server process (OS pid = 27138 ) for CommOutboundMgr
ServerLog ComponentUpdate 2 0000245b53e06913:0 2014-08-05 03:40:54 CommOutboundMgr INITIALIZED Component has initialized.
ServerLog ProcessExit 1 0000245d53e06913:0 2014-08-05 11:31:34 CommOutboundMgr 27138 SBL-OSD-02006 Process 27138 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 11:31:34 Created multithreaded server process (OS pid = 14727 ) for CommOutboundMgr
ServerLog ProcessExit 1 0000245d53e06913:0 2014-08-05 12:08:51 CommOutboundMgr 14727 SBL-OSD-02006 Process 14727 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 12:08:51 Created multithreaded server process (OS pid = 17797 ) for CommOutboundMgr
ServerLog ProcessExit 1 0000245d53e06913:0 2014-08-05 12:52:57 CommOutboundMgr 17797 SBL-OSD-02006 Process 17797 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 12:52:57 Created multithreaded server process (OS pid = 21434 ) for CommOutboundMgr
ServerLog ProcessExit 1 0000245d53e06913:0 2014-08-05 13:01:06 CommOutboundMgr 21434 SBL-OSD-02006 Process 21434 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 13:01:06 Created multithreaded server process (OS pid = 22020 ) for CommOutboundMgr
ServerLog ProcessExit 1 0000245d53e06913:0 2014-08-05 13:11:18 CommOutboundMgr 22020 SBL-OSD-02006 Process 22020 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 13:11:18 Created multithreaded server process (OS pid = 22845 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 15:05:11 CommOutboundMgr 22845 TERMINATED Process 22845 was terminated
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 15:15:42 CommOutboundMgr 3551 SBL-OSD-02006 Process 3551 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 15:15:42 Created multithreaded server process (OS pid = 6166 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 16:09:21 CommOutboundMgr 6166 SBL-OSD-02006 Process 6166 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 16:09:21 Created multithreaded server process (OS pid = 22642 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 16:27:46 CommOutboundMgr 22642 SBL-OSD-02006 Process 22642 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 16:27:46 Created multithreaded server process (OS pid = 24137 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 17:07:30 CommOutboundMgr 24137 SBL-OSD-02006 Process 24137 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 17:07:30 Created multithreaded server process (OS pid = 27361 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 17:27:54 CommOutboundMgr 27361 SBL-OSD-02006 Process 27361 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 17:27:54 Created multithreaded server process (OS pid = 28964 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 17:59:49 CommOutboundMgr 28964 SBL-OSD-02006 Process 28964 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 17:59:49 Created multithreaded server process (OS pid = 1556 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 19:18:49 CommOutboundMgr 1556 SBL-OSD-02006 Process 1556 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 19:18:49 Created multithreaded server process (OS pid = 7880 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 19:50:46 CommOutboundMgr 7880 SBL-OSD-02006 Process 7880 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 19:50:46 Created multithreaded server process (OS pid = 10522 ) for CommOutboundMgr
ServerLog ProcessExit 1 00044cf053e07436:0 2014-08-05 20:00:47 CommOutboundMgr 10522 SBL-OSD-02006 Process 10522 exited with error - Process exited because it received signal SIGABRT.
ServerLog ProcessCreate 1 0000245b53e06913:0 2014-08-05 20:00:47 Created multithreaded server process (OS pid = 11426 ) for CommOutboundMgr



The issue can NOT be reproduced at will, analyzing the call stack for the crashing thread on Solaris environment the majority of the CommOutboundMgr crashes shows:

/data01/siebel/siebsrvr/lib/libsslcosd.so'0x4ba60 [0xfe70ba60]
/lib/libc.so.1'__sighndlr+0xc [0xebc55b10]
/lib/libc.so.1'call_user_handler+0x370 [0xebc48f00]
/lib/libc.so.1'sigacthandler+0x58 [0xebc49208]
/lib/libc.so.1'strlen+0x18 [0xebbcc5b8] [Signal 11 (SEGV)]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cJTCPSocketRTCP_init_sockaddr6MpnLsockaddr_in_pkHHb_b_+0x74 [0xdf60a9d4]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cJTCPSocketLTCP_Connect6MpkHib_b_+0x54 [0xdf60ab14]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cDFTPOopenConnection6MnISSstring_pH21i_b_+0x164 [0xdf60ca04]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cNCSSFTPSessionHConnect6MrpnDFTP_rnISSstring_5555pH_i_+0xb4 [0xdf61a6b4]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cNCSSFTPSessionHSendMsg6MpnJSSCommMsg_pFpkH_ipH_i_+0x194 [0xdf61afd4]
/data01/siebel/siebsrvr/lib/libsscmftp.so'__1cYSCCommConnSessionServicePInvokeCommandEx6MrknISSstring_nPSCCommandTypeEx_rknNSCKVParamList__nHSSError__+0x90 [0xdf617b90]
/data01/siebel/siebsrvr/lib/libsscmftp.so'InvokeCommandEx+0x58 [0xdf613ef8]
/data01/siebel/siebsrvr/lib/libsscommsv.so'__1cQCSCDServiceProxyPInvokeCommandEx6MpkHnPSCCommandTypeEx_rknUCSSMapStringToString__l_+0x12c [0xee87df6c]
/data01/siebel/siebsrvr/lib/libsscommsv.so'__1cSCSSCommDeliveryMgrHSendMsg6MrknLMessageData_rnDstdElist4nKAttachment_n0CJallocator4n0D_____rknOCCFPropertySet_pnPIDriverCallback__I_+0xccc [0xee7f9eec]
/data01/siebel/siebsrvr/lib/libsscommsv.so'__1cQCSSSvcCOMMEngineMSendMessageW6MrknOCCFPropertySet__I_+0x9f8 [0xee859cd8]
/data01/siebel/siebsrvr/lib/libsscommsv.so'__1cQCSSSvcCOMMEngineODoInvokeMethod6MpkHrknOCCFPropertySet_r3_I_+0x3ec [0xee8132cc]
/data01/siebel/siebsrvr/lib/libsscfom.so'__1cKCSSServiceMInvokeMethod6MpkHrknOCCFPropertySet_r3_I_+0x338 [0xf92f70b8]
/data01/siebel/siebsrvr/lib/libsssabsvm.so'0x3b8b8 [0xfa37b8b8]
/data01/siebel/siebsrvr/lib/libsssabsvm.so'__1cMbsvmMTServerJBatchMain6MpvpnKSrmRequest__i_+0xe8 [0xfa37b0e8]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cKSmiRequestRCallBatchEntryPts6MpnMscrRecovInfo__i_+0xd0 [0x2205d0]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cKSmiRequestRCallBatchEntryPts6M_i_+0x98 [0x220398]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cKSmiRequestOProcessRequest6MlpnOSrmSisConnHndl_pnQCSSSISRequestMsg_LpCI_i_+0x5f0 [0x21f790]
/data01/siebel/siebsrvr/bin/siebmtshmw'0x214380 [0x224380]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cK_smiBatchQdDOProcessRequest6Fpv1r1_i_+0x34 [0x224d94]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cN_smiWorkQdDueuePProcessWorkItem6Mpv1r1_i_+0xd4 [0x21d134]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cN_smiWorkQdDueueKWorkerTask6Fpv_i_+0x314 [0x21cb54]
/data01/siebel/siebsrvr/bin/siebmtshmw'__1cQSmiThrdEntryFunc6Fpv_i_+0x494 [0x20a134]
/data01/siebel/siebsrvr/lib/libsslcosd.so'0x5d01c [0xfe71d01c]
/data01/siebel/siebsrvr/mw/lib/libmfc400su.so'__1cP_AfxThreadEntry6Fpv_I_+0x100 [0xfdfe5730]
/data01/siebel/siebsrvr/mw/lib/libkernel32.so'__1cIMwThread6Fpv_v_+0x23c [0xfcb017c8]
/lib/libc.so.1'_lwp_start+0x0 [0xebc55a10]

When inspecting the CommOutboundMgr log files for the crashing thread we could see that it was trying to use FTP driver to FTP a file. The customer has Marketing Campaign daily that FTP files to a FTP Server and it was confirmed that no files were been FTP

Checking the CommOutboundMgr log files for the FTP attempt that did not cause a crash we could see that FTP failed with error bellow:

ObjMgrLog Error 1 0000000353e06916:0 2014-08-14 16:09:31 (commdlv.cpp (600)) SBL-CMO-00152: Send  to , subject  had error

Because of the retry attempt a new Job is create for CommOutboundMgr and this attempt will keep going every 15 minutes for 24 hours. So this was also creating a high load on CommOutboundMgr for the FTP attempts.

The failure to upoad a file behavior could be reproduced with CommOutboundMgr in Solaris 11 and Windows Server 2012 when FTP a file for a Windows Server 2012 FTP Server.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.