My Oracle Support Banner

How to recover from a failed Linux Exadata DB Server dbnodeupdate or rollback (Doc ID 1952372.1)

Last updated on JANUARY 25, 2020

Applies to:

Oracle Exadata Storage Server Software - Version and later
Linux x86-64


When updates or rollbacks (possibly via fail, an Exadata database server may become unbootable. It also happens a database server does boot but fails to load the right modules or libraries after a broken or interrupted update, effectively making the system unusable. As a result it can happens users and administrators are unable to login to the system, even via the console. This document describes how to recover from a failing update or rollback on an Exadata DB server. This document only applies to X2 and later DB servers where logical volume management (lvm) is in use and where a backup was made to the inactive system partition prior the update. 

If the database server was configured with logical volume management and a backup was made prior to the update, there is a way to 'rollback' to that backup, by 'switching' system partitions. When switching system partitions, the active logical volume becomes inactive and the inactive (the backup) becomes active. In addition to the switching of active and inactive system partitions, the /boot directory belonging to the system partition being made the new active system partition needs to be restored. Also the Grub bootloader needs to be re-installed (for Grub1 only).

On systems where administrators can login and run, rollbacks should always be performed by On system where you cannot login, the process of switching system partitions, restoring /boot and bootloader reinstallation can only be done when booting from the diagnostic iso first. This note will guide the reader through the process of booting from diagnostic iso and restoring the relevant components.

Situations where the steps in this document can be used for:

Note: Failing updates to Exadata (Oracle Linux 6) or Oracle Linux 7 should in most cases rollback automatically. 


To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
 Boot the system from diagnostic iso (Grub 1 + Grub2)
 Switch filesystem labels (Grub 1 + Grub2)
 Restore /boot (Grub 1 + Grub 2)
 Install grub (Grub 1 only - this step can be skipped for Grub2)
 Install grub (Grub 2 only - this step can be skipped for Grub1)

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.