My Oracle Support Banner

Oracle Linux 7 server running RAC experiences hangs in the crsctl commands whenever it is run on the server (Doc ID 2589180.1)

Last updated on MARCH 15, 2021

Applies to:

Linux OS - Version Oracle Linux 7.0 and later
Linux x86-64

Symptoms

Oracle Linux 7 servers that ar part of a RAC cluster experience hangs with the crsctl command when it is run.


The server starts up with no issues as expected, however after running for sometime, the crsctl commands hangs, with the following seen from strace output from the hung crsctl command:

3648 poll([{fd=57, events=POLLIN}, {fd=59, events=POLLIN}, {fd=54, events=POLLIN}], 3, 0) = 0 (Timeout)
3648 times(NULL) = 429566660
3648 shutdown(58, SHUT_RDWR) = 0
3648 close(58) = 0
3648 shutdown(59, SHUT_RDWR) = 0
3648 close(59) = 0
3648 gettimeofday({1567208132, 523684}, NULL) = 0
3648 gettimeofday({1567208132, 523714}, NULL) = 0
3648 socketpair(AF_LOCAL, SOCK_STREAM, 0, [58, 59]) = 0
3648 ioctl(58, FIONBIO, [1]) = 0
3648 fcntl(58, F_GETFD) = 0
3648 fcntl(58, F_SETFD, FD_CLOEXEC) = 0
3648 ioctl(59, FIONBIO, [1]) = 0
3648 fcntl(59, F_GETFD) = 0
3648 fcntl(59, F_SETFD, FD_CLOEXEC) = 0
3648 times(NULL) = 429566660
3648 poll([{fd=57, events=POLLIN}, {fd=59, events=POLLIN}, {fd=54, events=POLLIN}], 3, -1) = 1 ([{fd=57, revents=POLLIN}])
3648 recvfrom(57, "p-\0\0\3\0\0\0\f\2\1\0\2\0\0\0\


The following was also logged to /var/log/messages when the crsctl command hung:

Aug 30 13:21:52 <hostname> abrt-hook-ccpp: Process 18304 (ocssd.bin) of user 54322 killed by SIGABRT - dumping core Aug 30 13:21:52 <hostname> abrt-hook-ccpp: Failed to create core_backtrace: waitpid failed: No child processes
Aug 30 13:21:52 <hostname> abrt-server: Executable '/u01/app/12.2.0/grid/bin/ocssd.bin' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Aug 30 13:21:52 <hostname> abrt-server: 'post-create' on '/var/spool/abrt/ccpp-2019-08-30-13:21:52-18304' exited with 1
Aug 30 13:21:52 <hostname> abrt-server: Deleting problem directory '/var/spool/abrt/ccpp-2019-08-30-13:21:52-18304'
Aug 30 13:26:00 <hostname> systemd: Starting Cleanup of Temporary Directories...
Aug 30 13:26:00 <hostname> systemd: Started Cleanup of Temporary Directories.
Aug 30 13:30:01 <hostname> systemd: Created slice User Slice of root.
Aug 30 13:30:01 <hostname> systemd: Started Session 3 of user root.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.