'rwdiag.bat' cannot locate Reports Servers in the Same Subnet (after servers were moved to a different Datacenter).
(Doc ID 2943213.1)
Last updated on AUGUST 29, 2023
Applies to:
Oracle Clinical - Version 5.1.0 and laterOracle Reports Developer - Version 11.1.2.2.0 and later
Information in this document applies to any platform.
Symptoms
1)
Unable to run a report when the report server is on a different FMW application server in the same environment (after moving servers to a new datacenter).
The 'RS_SUBMITFAILED' error is returned when submitting reports from the Oracle Clinical (OC) & Remote Data Capture (RDC) applications.
2)
It is possible to run reports when submitted to the same application server the user is connected to via the application i.e.
- user connects to Apps Server A > submits report from application to the reports server on Apps Server A => SUCCESS
- user connects to Apps Server A > submits report from application to the reports server on Apps Server B => FAIL
3)
Issue is only observed on the UAT environment (all OK on the PROD environment).
4)
Below changes were done to the three UAT Application servers (Virtual Machines):
- moved to a different datacenter
- IP Address changed
- configuration was updated to reflect new IP Address
5)
It was confirmed the thee UAT VM Application Servers are all on the same subnet (confirmed with command: ipconfig /all ):
- XXXXXXXXUATAPP1 (IP Adddress=xx.y.zzz.200)
- XXXXXXXXUATAPP2 (IP Adddress=xx.y.zzz.201)
- XXXXXXXXUATAPP3 (IP Adddress=xx.y.zzz.202)
6)
rwdiag.bat -findAll output confirmed the following on the problem UAT environment:
i.
uses: D:\oracle\Middleware\asinst_1\config\ReportsToolsComponent\ReportsTools\rwnetwork.conf
Broadcast mechanism used to locate servers
------------------------------------------
Channel address = 228.5.6.7
Channel port = 14021
ii.
RESULT: can only see its own standalone Reports Server: repxxxxxxxuatapp1
i.
uses: D:\oracle\Middleware\asinst_1\config\ReportsToolsComponent\ReportsTools\rwnetwork.conf
Broadcast mechanism used to locate servers
------------------------------------------
Channel address = 228.5.6.7
Channel port = 14021
ii.
RESULT: can only see its own standalone Reports Server: repxxxxxxxuatapp2
i.
uses: D:\oracle\Middleware\asinst_1\config\ReportsToolsComponent\ReportsTools\rwnetwork.conf
Broadcast mechanism used to locate servers
------------------------------------------
Channel address = 228.5.6.7
Channel port = 14021
ii.
RESULT: can only see its own standalone Reports Server: repxxxxxxxuatapp3
7)
rwdiag.bat -findAll output confirmed the following on the working PROD environment:
uses: D:\oracle\Middleware\asinst_1\config\ReportsToolsComponent\ReportsTools\rwnetwork.conf
Broadcast mechanism used to locate servers
------------------------------------------
Channel address = 228.5.6.7
Channel port = 14021
ii.
RESULT: Reports Servers on different Application Servers can be seen e.g.
Production Server XXXXXXXXPRDAPP2 can see:
- repxxxxxxxprdapp2
- repxxxxxxxprdapp3
- repxxxxxxxprdapp4
Production Server XXXXXXXXPRDAPP3 can see:
- repxxxxxxxprdapp2
- repxxxxxxxprdapp3
- repxxxxxxxprdapp4
etc
8)
Each UAT server has one Network Interface Card (NIC) and do not have multiple NIC cards.
9)
The documented multicast connectivity test ( https://docs.oracle.com/cloud/latest/fmw122100/COHAG/GUID-218DE54A-A627-45DB-A5E9-C643613C3DD9.htm#COHAG182 ) was performed on two application servers in the working PROD environment to confirm multicast packets can travel across servers as follows (note: 228.5.6.7:14021 is specified for the group in the multicast-test.cmd command line. This is used because this is what appears in the rwnetwork.conf file i.e. <multicast channel="228.5.6.7" port="14021" timeout="1000"/> )
i.
Sanity check => run following on the PROD XXXXXXXXPRDAPP2/ XXXXXXXXPRDAPP3 servers and ensure reports servers can be seen on each server:
D:\oracle\Middleware\asinst_1\config\reports\bin>rwdiag.bat -findAll
ii.
Perform below steps on both servers at the same time so both commands are therefore running in parallel (ensure the correct path for JAVA_HOME is specified):
XXXXXXXXPRDAPP2:
- Open CMD window
- FMW 11g: navigate to: C:\Oracle\Middleware\coherence_3.7\bin
- FMW 12c: navigate to: C:\Oracle\Middleware\Oracle_Home\coherence\bin
- Set JAVA_HOME= < SPECIFY YOUR JDK HOME PATH e.g. C:\oracle\jdk180 ) >
- Run command: multicast-test.cmd -group 228.5.6.7:14021
XXXXXXXXPRDAPP3:
- Open CMD window
- FMW 11g: navigate to: C:\Oracle\Middleware\coherence_3.7\bin
- FMW 12c: navigate to: C:\Oracle\Middleware\Oracle_Home\coherence\bin
- Set JAVA_HOME= < SPECIFY YOUR JDK HOME PATH e.g. C:\oracle\jdk180 ) >
- Run command: multicast-test.cmd -group 228.5.6.7:14021
iii.
Let the above run for about a minute on each server (in parallel) and then cancel the command using CTRL+C.
RESULT:
ref: rwdiag.bat -findAll command: CAN see reports servers on own server / other servers in same subnet:
XXXXXXXXPRDAPP2:
(1) Name = repxxxxxxxxprdapp2 : Type = server : Host = XXXXXXXXPRDAPP2.DOMAIN
(2) Name = rptsvr_xxxxxxxxprdapp2_asinst_ : Type = server : Host = XXXXXXXXPRDAPP2.DOMAIN
(3) Name = repxxxxxxxxcprdapp3 : Type = server : Host = XXXXXXXXPRDAPP3.DOMAIN
(4) Name = rptsvr_xxxxxxxxprdapp3_asinst_ : Type = server : Host = XXXXXXXXPRDAPP3.DOMAIN
XXXXXXXXPRDAPP3:
(1) Name = repxxxxxxxxprdapp3 : Type = server : Host = XXXXXXXXPRDAPP3.DOMAIN
(2) Name = rptsvr_xxxxxxxxprdapp3_asinst_ : Type = server : Host = XXXXXXXXPRDAPP3.DOMAIN
(3) Name = repxxxxxxxxprdapp2 : Type = server : Host = XXXXXXXXPRDAPP2.DOMAIN
(4) Name = rptsvr_xxxxxxxxprdapp2_asinst_ : Type = server : Host = XXXXXXXXPRDAPP2.DOMAIN
iv.
ref: multicast-test.cmd -group 228.5.6.7:14021 command: CAN send multicast packets to self and across servers in same subnet:
XXXXXXXXPRDAPP2:
Thu Mar 02 05:50:58 CST 2023: Sent packet 1 containing 1468 bytes.
Thu Mar 02 05:50:58 CST 2023: Received test packet 1 from self (sent 16ms ago).
Thu Mar 02 05:51:00 CST 2023: Sent packet 2 containing 1468 bytes.
Thu Mar 02 05:51:00 CST 2023: Received test packet 2 from self
Thu Mar 02 05:51:00 CST 2023: Received test packet 1 from ip=XXXXXXXXPRDAPP3/xx.yy.z.178, group=/228.5.6.7:14021, ttl=4 containing 1468 bytes.
XXXXXXXXPRDAPP3:
Thu Mar 02 05:51:01 CST 2023: Sent packet 1 containing 1468 bytes.
Thu Mar 02 05:51:01 CST 2023: Received test packet 1 from self
Thu Mar 02 05:51:02 CST 2023: Received test packet 3 from ip=XXXXXXXXPRDAPP2/xx.yy.z.177, group=/228.5.6.7:14021, ttl=4 containing 1468 bytes.
Thu Mar 02 05:51:03 CST 2023: Sent packet 2 containing 1468 bytes.
Thu Mar 02 05:51:03 CST 2023: Received test packet 2 from self
10)
The above step was then performed on two application servers in the problem UAT env where it was determined both servers:
- CAN only see their own respective reports servers
- CANNOT see the reports server on the other server
i.e.
i.
ref: rwdiag.bat -findAll command: CANNOT see other reports servers / can only see their own reports servers on their respective servers:
XXXXXXXXUATAPP2:
(1) Name = repxxxxxxxxuatapp2 : Type = server : Host = XXXXXXXXUATAPP2.DOMAIN
(2) Name = rptsvr_xxxxxxxxuatapp2_asinst_ : Type = server : Host = XXXXXXXXUATAPP2.DOMAIN
(3) Name = rep_wls_reports_xxxxxxxxuatapp2_asinst_1 : Type = server : Host = XXXXXXXXUATAPP2.DOMAIN
XXXXXXXXUATAPP3:
(1) Name = rep_wls_reports_xxxxxxxxuatapp3_asinst_1 : Type = server : Host = XXXXXXXXUATAPP3.DOMAIN
(2) Name = rptsvr_xxxxxxxxuatapp3_asinst_ : Type = server : Host = XXXXXXXXUATAPP3.DOMAIN
(3) Name = repxxxxxxxxuatapp3 : Type = server : Host = XXXXXXXXUATAPP3.DOMAIN
ii.
ref: multicast-test.cmd -group 228.5.6.7:14021 command: CANNOT send multicast packets across servers in the same subnet:
XXXXXXXXUATAPP2:
Thu Mar 02 07:08:52 EST 2023: Sent packet 1 containing 1468 bytes.
Thu Mar 02 07:08:52 EST 2023: Received test packet 1 from self
Thu Mar 02 07:08:54 EST 2023: Sent packet 2 containing 1468 bytes.
Thu Mar 02 07:08:54 EST 2023: Received test packet 2 from self
XXXXXXXXUATAPP3:
Thu Mar 02 07:09:02 EST 2023: Sent packet 1 containing 1468 bytes.
Thu Mar 02 07:09:02 EST 2023: Received test packet 1 from self (sent 16ms ago).
Thu Mar 02 07:09:04 EST 2023: Sent packet 2 containing 1468 bytes.
Thu Mar 02 07:09:04 EST 2023: Received test packet 2 from self
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |
| References |