GI failed to start with "No space left on device" error although the filesystem has a large amount of free space

(Doc ID 2346416.1)

Last updated on JANUARY 12, 2018

Applies to:

Oracle Database - Standard Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.

Symptoms

Starting GI failed.

The ohasd.trc shows that the filesystem ran out of the space

2017-12-27 09:40:27.602
[ohasd(2531)]CRS-10000:CLSU-00100: Operating System function: mkdir failed with error data: 28
CLSU-00101: Operating System error message: No space left on device
CLSU-00103: error location: authprep6
CLSU-00104: additional error information: failed to make dir /u01/app/11.2.0/grid/auth/ohasd/<node name>/A9553324

 However, the filesystem had a plenty of free space.

df -k  (the / filesystem has the GI HOME and has 27 GB of free space)

Filesystem                                                       1K-blocks                          Used                          Available                      Use%                                  Mounted on
/dev/mapper/vg_ctnodo01-lv_root                       86347384                         54466476                   27513524                    67%                                    /
tmpfs                                                              8095496                           0                               8095496                           0%                                     /dev/shm
/dev/xvdb1                                                      495844                             55607                        414637                            12%                                    /boot
/dev/xvde1                                                      51613016                          498508                      48492704                        2%                                      /dbvisit


System Messages file may also show "No space left on device" error. 

For this case, both GI HOME and /var reside in the / file system.

Dec 24 09:09:24 ctnodo01 abrt[33900]: Can't create directory '/var/spool/abrt/ccpp-2017-12-24-09:09:24-33872.new': No space left on device
......
Dec 28 08:59:53 ctnodo01 gdm-binary[2272]: WARNING: Cannot write PID file /var/run/gdm.pid: possibly out of diskspace: No space left on device
Dec 28 08:59:53 ctnodo01 gdm-binary[2272]: CRITICAL: could not create display access file: Unable to open '/var/run/gdm/auth-for-gdm-ArmnEO/database': No space left on device
Dec 28 09:00:05 ctnodo01 gdm-binary[2272]: CRITICAL: could not create display access file: Unable to open '/var/run/gdm/auth-for-gdm-RNjQ9K/database': No space left on device
Dec 28 09:00:17 ctnodo01 gdm-binary[2272]: CRITICAL: could not create display access file: Unable to open '/var/run/gdm/auth-for-gdm-bEGePj/database': No space left on device

 

Changes

 None

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms