[OLVM] Unable To Migrate VMs From OL7.7 To OL7.8 VDSM nodes. (Doc ID 2742344.1)

Last updated on OCTOBER 01, 2021

Applies to:

Symptoms

Some VDSM nodes in OLVM environment were upgraded to Oracle Linux 7.8 from Oracle Linux 7.7 as part of rolling upgrade. After this, migration of VMs from OL 7.7 VDSM nodes to OL 7.8 VDSM nodes fails with the following error:

Failed Migratin VM <VM_name> to Host <hostname>.

Issue can be reproduced at will with the following steps:

1. Upgrade a KVM host (VDSM node) from OL 7.7 to OL 7.8.
2. Attempt to live migrate a VM from OL7.7 host over to OL7.8 host.

Engine logs the following error:

2020-06-10 08:02:15,850Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-5446) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: <VM_name>, Source: <Source_VDSM_nodename>, Destination: oralvm01-dc2.medirectbank.mt)

VDSM node logs the following in vdsm.log:

2020-06-10 08:03:13,402+0000 ERROR (migsrc/ab8fc9e5) [virt.vm] (vmId='ab8fc9e5-c1e3-403f-9443-0a9814695bd1') internal error: child reported (status=125): unable to open /rhev/data-center/mnt/10.168.100.40:_iso/caa7ef44-6d7d-4159-b662-120aadb8745f/images/11111111-1111-1111-1111-111111111111/V980739-01.iso: Permission denied (migration:282)
...
2020-06-10 08:03:15,520+0000 ERROR (migsrc/ab8fc9e5) [virt.vm] (vmId='ab8fc9e5-c1e3-403f-9443-0a9814695bd1') Failed to migrate (migration:450)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 431, in _regular_run
time.time(), migrationParams, machineParams
File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 505, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 591, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 525, in _perform_migration
self._migration_flags)
File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1781, in migrateToURI3
if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self)
libvirtError: internal error: child reported (status=125): unable to open /rhev/data-center/mnt/10.168.100.40:_iso/caa7ef44-6d7d-4159-b662-120aadb8745f/images/11111111-1111-1111-1111-111111111111/V980739-01.iso: Permission denied

Changes

Some of VDSM nodes (KVM hosts) were upgraded to OL 7.8 from OL 7.7.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.