The Admin Server for Oracle Traffic Directory Domain Takes Long Time to Start (Doc ID 2359400.1)

Last updated on NOVEMBER 06, 2023

Applies to:

Symptoms

The Admin Server takes over 30 minutes to start. After it finally starts up, logging into the Fusion Middleware Control Control (/em) and browsing through pages also appears to be very slow.

strace is taken at the time of the admin server startup and from reviewing the trace output it can be seen the admin server spends tremendous time going through WebGate logs (an Oracle Access Manager plugin configured on OTD). The below excerpt shows that it takes 16 seconds to read just one log file:

5207 10:50:24.062116 lstat("<PATH>/otd_ConfigName_MachineName/webgate.log.20171030.141037.215337", {st_dev=makedev(0, 27), st_ino=14989, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1577, st_gid=500, st_blksize=131072, st_blocks=102495, st_size=52404789, st_atime=2018/01/31-11:34:16, st_mtime=2017/10/30-10:09:37, st_ctime=2017/10/30-10:10:37}) = 0
5207 10:50:24.062180 gettimeofday({1517845824, 62201}, NULL) = 0
5207 10:50:24.062237 stat("<PATH>/otd_ConfigName_MachineName/webgate.log.20171030.141037.215337", {st_dev=makedev(0, 27), st_ino=14989, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1577, st_gid=500, st_blksize=131072, st_blocks=102495, st_size=52404789, st_atime=2018/01/31-11:34:16, st_mtime=2017/10/30-10:09:37, st_ctime=2017/10/30-10:10:37}) = 0
5207 10:50:24.062324 open("<DOMAIN_HOME>/config/fmwconfig/components/OTD/logs/otd_ConfigName_MachineName/webgate.log.20171030.141037.215337", O_RDONLY <unfinished ...>
4538 10:50:24.062468 <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
4538 10:50:24.062497 futex(0x7f02f2421228, FUTEX_WAKE_PRIVATE, 1) = 0
4538 10:50:24.062534 clock_gettime(CLOCK_MONOTONIC, {935544, 381409549}) = 0
4538 10:50:24.062576 clock_gettime(CLOCK_MONOTONIC, {935544, 381450823}) = 0
4538 10:50:24.062611 clock_gettime(CLOCK_MONOTONIC, {935544, 381485319}) = 0
4538 10:50:24.062645 clock_gettime(CLOCK_MONOTONIC, <unfinished ...>
5207 10:50:24.062683 <... open resumed> ) = 826
....
...
5207 10:50:24.107459 read(826, "NNER INFORMATION ***************"..., 128) = 128
5207 10:50:24.107500 read(826, "2/webgate/otd\nWebgateInstanceDir"..., 128) = 128
5207 10:50:24.107538 read(826, "/config/\nAgentType=WebGate\nWebga"..., 128) = 128
5207 10:50:24.107575 read(826, "e=Oracle Traffic Director/12.2.1"..., 128) = 128
5207 10:50:24.107611 read(826, "8:51:42 PDT 2017 x86_64\n********"..., 128) = 128
....
...
5207 10:50:40.868382 read(826, "83351\t18890\t18914\tCONN_MGMT\tERRO"..., 128) = 128
5207 10:50:40.868419 read(826, "src/watcher_thread.cpp:361\t\"Open"..., 128) = 128
5207 10:50:40.868456 read(826, "priority=1}{_debug=false}{_host="..., 128) = 128
5207 10:50:40.868494 read(826, "778}{_closedTime=0}{_retries=440"..., 128) = 53
5207 10:50:40.868531 read(826, "", 128) = 0
5207 10:50:40.868570 close(826) = 0

Changes

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.