ASM disk addition failing on DOM U level as gnttab_max_frames maxed out at DOM 0 level (Doc ID 2554409.1)

GMON querying group 1 at 153 for pid 16, osid 6595
NOTE: cache opening disk 28 of grp 1: DATA_0028 path:/dev/asm-disk29
GMON querying group 1 at 154 for pid 16, osid 6595
Wed Jun 05 14:23:52 2019
SUCCESS: refreshed membership for 1/0x6099453f (DATA)
Wed Jun 05 14:23:52 2019
NOTE: starting rebalance of group 1/0x6099453f (DATA) at power 1
Starting background process ARB0
Wed Jun 05 14:23:52 2019
SUCCESS: ALTER DISKGROUP data ADD DISK '/dev/asm-disk29' NAME DATA_0028
Wed Jun 05 14:23:52 2019
ARB0 started with pid=27, OS id=27005
NOTE: assigning ARB0 to group 1/0x6099453f (DATA) with 1 parallel I/O
NOTE: Attempting voting file refresh on diskgroup DATA
Wed Jun 05 14:24:03 2019
SQL> ALTER DISKGROUP data ADD DISK '/dev/asm-disk30' NAME DATA_0029
Wed Jun 05 14:24:03 2019
NOTE: stopping process ARB0
Wed Jun 05 14:24:06 2019
NOTE: rebalance interrupted for group 1/0x6099453f (DATA)

Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.081539] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=17
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.082098] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.082104] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.083343] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.084407] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.085222] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.086162] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.086926] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.117889] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:23:58 xxxxxxxxxNN kernel: [3395057.118988] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:24:03 xxxxxxxxxNN kernel: [3395062.096922] gnttab_expand: 2459 callbacks suppressed

Jun 5 14:24:03 xxxxxxxxxNN kernel: [3395062.105521] xen:grant_table: xen/grant-table: max_grant_frames reached cur=64 extra=1 limit=64 gnttab_free_count=0 req_entries=15
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.150258] INFO: task oracle_23940_+a:23940 blocked for more than 120 seconds.
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.151320] Not tainted 4.1.12-124.23.2.el6uek.x86_64 #2
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.151806] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152683] oracle_23940_+a D ffff883b9d2d8480 0 23940 15611 0x00000080
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152687] ffff881119c435e8 0000000000000082 ffffffff816f669a ffff881daba27000
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152690] ffffffff816f66fc ffff881119c44000 ffff883b9d2d8480 7fffffffffffffff
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152692] ffff881119c43808 ffffe8ffffacaf80 ffff881119c43608 ffffffff816ec5c7
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152695] Call Trace:
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152703] [] ? xen_hvm_callback_vector+0xba/0x1a0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152705] [] ? xen_hvm_callback_vector+0x11c/0x1a0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152710] [] schedule+0x37/0x90
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152712] [] ? xen_hvm_callback_vector+0xe4/0x1a0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152714] [] schedule_timeout+0x24c/0x2d0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152718] [] ? ___ratelimit+0x8b/0x110
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152723] [] ? try_to_grab_pending+0xc1/0x160
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152726] [] ? xen_clocksource_get_cycles+0x15/0x20
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152730] [] ? ktime_get+0x41/0xc0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152733] [] io_schedule_timeout+0xa4/0x110
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152736] [] bt_get+0x137/0x1d0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152740] [] ? wait_woken+0x90/0x90
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152742] [] blk_mq_get_tag+0xa7/0xd0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152744] [] ? xen_clocksource_read+0x15/0x20
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152747] [] __blk_mq_alloc_request+0x1b/0x1f0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152749] [] blk_mq_map_request+0x1c2/0x250
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152753] [] ? part_round_stats+0x4f/0x70
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152755] [] blk_sq_make_request+0x87/0x3b0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152757] [] ? generic_make_request_checks+0x217/0x450
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152760] [] generic_make_request+0xc6/0x1d0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152762] [] submit_bio+0x78/0x180
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152766] [] do_blockdev_direct_IO+0x22bc/0x2f50
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152769] [] ? I_BDEV+0x10/0x10
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152772] [] __blockdev_direct_IO+0x43/0x50
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152774] [] blkdev_direct_IO+0x58/0x80
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152778] [] generic_file_direct_write+0xb0/0x180
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152780] [] __generic_file_write_iter+0xb4/0x1d0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152783] [] ? bd_unlink_disk_holder+0xf0/0xf0
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152785] [] blkdev_write_iter+0x7c/0x110
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152789] [] aio_run_iocb+0x27c/0x300
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152792] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152794] [] ? __schedule+0x24a/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152796] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152798] [] ? __schedule+0x24a/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152801] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152803] [] ? __schedule+0x24a/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152805] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152807] [] ? __schedule+0x24a/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152809] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152812] [] ? __schedule+0x24a/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152816] [] ? kmem_cache_alloc+0x222/0x250
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152818] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152820] [] ? __schedule+0x23e/0x810
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152823] [] do_io_submit+0x288/0x530
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152825] [] ? system_call_after_swapgs+0xe9/0x190
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152828] [] SyS_io_submit+0x10/0x20
Jun 5 14:27:42 xxxxxxxxxNN kernel: [3395281.152830] [] system_call_fastpath+0x18/0xd8
Jun 5 14:29:42 xxxxxxxxxNN kernel: [3395401.152253] INFO: task oracle_23940_+a:23940 blocked for more than 120 seconds.
Jun 5 14:29:42 xxxxxxxxxNN kernel: [3395401.153153] Not tainted 4.1.12-124.23.2.el6uek.x86_64 #2

File_name:: messages