Dbnodeupdate.sh Finishes With Error: System Not On Expected Exadata Release Or Kernel, Exiting

(Doc ID 2007282.1)

Last updated on JULY 13, 2015

Applies to:

Oracle Exadata Storage Server Software - Version 12.1.2.1.1 and later
Linux x86-64

Symptoms

Updating an Exadata dbnode failed with an error during the final clean-up dbnodeupdate execution (./dbnodeupdate.sh -c):
<<

Continue ? [y/n]
y
  (*) 2015-05-05 15:00:31: Verifying GI and DB's are shutdown
  (*) 2015-05-05 15:00:33: Verifying firmware updates/validations. Maximum wait time: 60 minutes.
  (*) 2015-05-05 15:00:33: If the node reboots during this firmware update/validation, re-run './dbnodeupdate.sh -c' after the node restarts..
  (*) 2015-05-05 15:00:33: Collecting console history for diag purposes

  ERROR: System not on expected Exadata release or kernel, exiting


  ERROR: Correct error, or to override run: ./dbnodeupdate.sh -c -q -t 12.1.2.1.1.150316.2

>>

During the first executiong of the dbnodeupdate.sh all new packages were installed and the node rebooted as expected, but the dbnode rebooted with the wrong kernel (the old one) because grub.conf was not updated.

Here's the information after the reboot:
[root cellos]# imageinfo

Kernel version: 2.6.39-400.243.1.el6uek.x86_64 #1 SMP Wed Nov 26 09:15:35 PST 2014 x86_64
Image version: 12.1.2.1.1.150316.2
Image activated: 2015-05-05 14:50:35 +0200
Image status: success
System partition on device: /dev/mapper/VGExaDb-LVDbSys1

[root cellos]# imagehistory
Version : 12.1.2.1.0.141206.1
Image activation date : 2015-02-01 15:38:34 -0800
Imaging mode : fresh
Imaging status : success

Version : 12.1.2.1.1.150316.2
Image activation date : 2015-05-05 14:50:35 +0200
Imaging mode : patch
Imaging status : success

[root cellos]# ls -l /boot/
total 36976
-rw-r----- 1 root root 115393 26 nov 18:36 config-2.6.39-400.243.1.el6uek.x86_64
-rw-r--r-- 1 root root 115393 12 mar 02:36 config-2.6.39-400.248.3.el6uek.x86_64
drwxrwxr-x 2 root root 4096 5 mag 15:14 grub
-rw-r----- 1 root root 36 1 feb 15:57 I_am_hd_boot
-rw------- 1 root root 7913015 5 mag 14:45 initramfs-2.6.39-400.243.1.el6uek.x86_64.img
-rw------- 1 root root 7912143 5 mag 14:42 initramfs-2.6.39-400.248.3.el6uek.x86_64.img
-rw------- 1 root root 8249115 2 feb 00:36 initrd-2.6.39-400.243.1.el6uek.x86_64kdump.img
drwx------ 2 root root 16384 1 feb 15:57 lost+found
-rw-r----- 1 root root 190627 26 nov 18:38 symvers-2.6.39-400.243.1.el6uek.x86_64.gz
-rw-r--r-- 1 root root 190678 12 mar 02:43 symvers-2.6.39-400.248.3.el6uek.x86_64.gz
-rw-r----- 1 root root 2461543 26 nov 18:36 System.map-2.6.39-400.243.1.el6uek.x86_64
-rw-r--r-- 1 root root 2462053 12 mar 02:36 System.map-2.6.39-400.248.3.el6uek.x86_64
-rwxr-x--- 1 root root 4103200 26 nov 18:36 vmlinuz-2.6.39-400.243.1.el6uek.x86_64
-rwxr-xr-x 1 root root 4104672 12 mar 02:36 vmlinuz-2.6.39-400.248.3.el6uek.x86_64

[root cellos]# uname -a
Linux exadb01 2.6.39-400.243.1.el6uek.x86_64 #1 SMP Wed Nov 26 09:15:35 PST 2014 x86_64 x86_64 x86_64 GNU/Linux

 

As seen above the current kernel is 2.6.39-400.243.1.el6uek.x86_64 (the old one), but the new installed kernel is 2.6.39-400.248.3.el6uek.x86_64, all new packages are there.

Here's what the grub.conf file shows:

[root cellos]# cat /etc/grub.conf
# Copyright (c) 2008, 2014, Oracle and/or its affiliates. All rights reserved.

default=0
timeout=5
hiddenmenu
#splashimage=(hd0,0)/grub/oracle.xpm.gz
color white/blue yellow/magenta
password --md5 $1$Hdnwr/$Q5VoiZeTJrmNQsJnH9oFy.

title Exadata_DBM: LINUX_BOOT_0
root (hd0,0)
  kernel /vmlinuz-2.6.39-400.243.1.el6uek.x86_64 root=LABEL=DBSYS bootarea=dbsys bootfrom=BOOT ro loglevel=7 panic=60 debug pci=noaer log_buf_len=1m nmi_watchdog=0 nomce transparent_hugepage=never rd_NO_PLYMOUTH audit=1 console=tty1 console=ttyS0,115200n8 crashkernel=380M@128M numa=on
initrd /initramfs-2.6.39-400.243.1.el6uek.x86_64.img

 

Notice the new kernel 2.6.39-400.248.3.el6uek.x86_64 is not listed.

Changes

 Updating the dbnode using dbnodeupdate.sh

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms