All Processes that Invoke an FTP Adapter to the same FTP server start failing and enter recovery without any fatal error message being logged (Doc ID 1290241.1)

Last updated on JULY 08, 2013

Applies to:

Oracle(R) BPEL Process Manager - Version 11.1.1.2.0 to 11.1.1.3.0 [Release 11gR1]
Information in this document applies to any platform.

Symptoms

You have developed a composite application, that is deployed to an Oracle Fusion Middleware 11g Release 1 (11.1.1.2.0 or 11.1.1.3.0 patch level).

Your application processing uses the BPEL Process Manager which makes outbound calls to the FTP Adapter to create files on an FTP/SFTP server and at runtime everything runs fine for very long periods (potentially several days). However, you are also seeing repeated periods of instability that re-occur quite  frequently and are characterized by the following symptoms:

  1. All instances of various BPEL processes fail to complete and go into recovery.
  2. None of the log messages related to the failed BPEL process indicate any significant error occurred during processing so there is no obvious candidate for the failure.
  3. All attempts to manually recover the failed process fail to make any progress.
  4. If the SOA managed server is stopped and restarted then the failed processes automatically recover with no other corrective action needed.
  5. Prior to stopping the SOA managed server you invoke a thread dump that shows one of  more threads with the following stack trace:

    "orabpel.invoke.pool-3.thread-16" prio=3 tid=0x000000010d88f000 nid=0x1b0 runnable [0xfffffffa452f8000]
    java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at java.net.SocketInputStream.read(SocketInputStream.java:182)
    at com.maverick.ssh.SshConnector.A(Unknown Source)
    at com.maverick.ssh.SshConnector.connect(Unknown Source)
    at oracle.tip.adapter.ftp.SshImpl.SSHSessionImpl.
       setUpPasswordSocketConnection(SSHSessionImpl.java:187)
    at oracle.tip.adapter.ftp.SshImpl.SSHSessionImpl.
       <init>(SSHSessionImpl.java:123)
    at oracle.tip.adapter.ftp.SshImpl.SshImplFactory.
       getSshImpl(SshImplFactory.java:26)
    at oracle.tip.adapter.ftp.SFTPManagedConnection.
       setupSftpConnection(SFTPManagedConnection.java:136)
    at oracle.tip.adapter.ftp.SFTPManagedConnection.
       <init>(SFTPManagedConnection.java:66)
    at oracle.tip.adapter.ftp.FTPManagedConnectionFactory.
       createManagedConnection(FTPManagedConnectionFactory.java:181)
    at weblogic.connector.security.layer.AdapterLayer.
       createManagedConnection(AdapterLayer.java:802)
    at weblogic.connector.outbound.ConnectionFactory.
       createResource(ConnectionFactory.java:91)
    at weblogic.common.resourcepool.ResourcePoolImpl.
       makeResources(ResourcePoolImpl.java:1180)
    at weblogic.common.resourcepool.ResourcePoolImpl.
       reserveResourceInternal(ResourcePoolImpl.java:395)
    at weblogic.common.resourcepool.ResourcePoolImpl.
       reserveResource(ResourcePoolImpl.java:332)
    at weblogic.common.resourcepool.ResourcePoolImpl.
       reserveResource(ResourcePoolImpl.java:315)
    at weblogic.connector.outbound.ConnectionPool.
       reserveResource(ConnectionPool.java:596)
    at weblogic.common.resourcepool.ResourcePoolImpl.
       reserveResource(ResourcePoolImpl.java:309)
    at weblogic.connector.outbound.ConnectionManagerImpl.
       getConnectionInfo(ConnectionManagerImpl.java:352)
    at weblogic.connector.outbound.ConnectionManagerImpl.
       getConnection(ConnectionManagerImpl.java:307)
    at weblogic.connector.outbound.ConnectionManagerImpl.
       allocateConnection(ConnectionManagerImpl.java:115)
    at oracle.tip.adapter.ftp.FTPConnectionFactory.
       getConnection(FTPConnectionFactory.java:103)
    at oracle.integration.platform.blocks.adapter.fw.jca.cci.
       JCAConnectionManager$JCAConnectionPool.
       createJCAConnection(JCAConnectionManager.java:1355)
    at oracle.integration.platform.blocks.adapter.fw.jca.cci.
       JCAConnectionManager$JCAConnectionPool$JCAConnection.
       init(JCAConnectionManager.java:1041)
    ...
    at com.collaxa.cube.engine.ejb.impl.bpel.BPELDeliveryBean_
       5k948i_ICubeDeliveryLocalBeanImpl.handleInvoke
      (BPELDeliveryBean_5k948i_ICubeDeliveryLocalBeanImpl.java:828)
    at com.collaxa.cube.engine.dispatch.message.invoke.
       InvokeInstanceMessageHandler.
       handle(InvokeInstanceMessageHandler.java:35)
    at com.collaxa.cube.engine.dispatch.DispatchHelper.
       handleMessage(DispatchHelper.java:142)
    at com.collaxa.cube.engine.dispatch.BaseDispatchTask.
       run(BaseDispatchTask.java:82)
    at java.util.concurrent.ThreadPoolExecutor$Worker.
       runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.
       run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
  6. The thread with the above signature fails to make any progress or movement in several successive thread dumps and remains in this condition until the SOA managed server is restarted.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms