OLVM: Failed to create Gluster Storage Bricks with error message "Error While Executing Action Create Brick Internal Engine Error" (Doc ID 2869809.1)

Last updated on OCTOBER 01, 2024

Applies to:

Symptoms

On Oracle Linux Virtualization Manager(OLVM), the system is failing to create the Gluster Storage bricks. See the details below.

/var/log/ovirt-engine/engine.log:
2022-05-13 23:09:54,486+07 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-16) [09bd36de-0ceb-47f0-a923-72192303397a] EVENT_ID: CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick on host OLVM-KVM-HOSTNx of cluster MY-TEST-CLUSTER-DC.
2022-05-13 23:16:25,513+07 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-435) [] Failed in 'GetGlusterHookContentVDS' method
2022-05-13 23:16:25,518+07 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-435) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM OLVM-KVM-HOSTNx command GetGlusterHookContentVDS failed: Internal JSON-RPC error: {'reason': "'utf-8' codec can't decode byte 0xc8 in position 1: unexpected end of data"}
2022-05-13 23:16:25,518+07 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHookContentVDSCommand] (EE-ManagedThreadFactory-engine-Thread-435) [] Command 'GetGlusterHookContentVDSCommand(HostName = VDSM OLVM-KVM-HOSTNx, GlusterHookVDSParameters:{hostId='25f6b349-8e9d-4b8e-9ffd-096890687b0a'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetGlusterHookContentVDS, error = Internal JSON-RPC error: {'reason': "'utf-8' codec can't decode byte 0xc8 in position 1: unexpected end of data"}, code = -32603
2022-05-13 23:16:25,518+07 ERROR [org.ovirt.engine.core.bll.gluster.GlusterHookSyncJob] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-56) [] Exception in sync: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetGlusterHookContentVDS, error = Internal JSON-RPC error: {'reason': "'utf-8' codec can't decode byte 0xc8 in position 1: unexpected end of data"}, code = -32603 (Failed with error unexpected and code 16)
2022-05-13 23:16:25,537+07 ERROR [org.ovirt.engine.core.bll.gluster.GlusterGeoRepSyncJob] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-80) [] VDS error Geo Rep status failed: rc=2 out=() err=['Command {self.cmd} failed with rc={self.rc} out={self.out!r} err={self.err!r}']
2022-05-13 23:19:56,593+07 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (default task-19) [9091f2b3-f3ef-4fdc-b1d6-e165d12714c0] Exception: Task Create volume groups failed to execute. Please check logs for more details: /var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20220513231950-192.168.2.53-9091f2b3-f3ef-4fdc-b1d6-e165d12714c0.log
2022-05-13 23:19:56,616+07 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-19) [9091f2b3-f3ef-4fdc-b1d6-e165d12714c0] Command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed: Task Create volume groups failed to execute. Please check logs for more details: /var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20220513231950-192.168.2.53-9091f2b3-f3ef-4fdc-b1d6-e165d12714c0.log
2022-05-13 23:19:56,616+07 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-19) [9091f2b3-f3ef-4fdc-b1d6-e165d12714c0] Exception: org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerCallException: Task Create volume groups failed to execute. Please check logs for more details: /var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible-20220513231950-192.168.2.53-9091f2b3-f3ef-4fdc-b1d6-e165d12714c0.log
2022-05-13 23:19:56,621+07 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-19) [9091f2b3-f3ef-4fdc-b1d6-e165d12714c0] Transaction rolled-back for command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'.
2022-05-13 23:19:56,626+07 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-19) [9091f2b3-f3ef-4fdc-b1d6-e165d12714c0] EVENT_ID: CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick brick1 on host OLVM-KVM-HOSTNx of cluster

/var/log/ovirt-engine/brick-setup/ovirt-gluster-brick-ansible********.log
2022-05-13 23:19:56 ICT - fatal: [xxx.xxx.xxx.xxx]: FAILED! => {"changed": false, "err": " Cannot use /dev/md125: device is rejected by filter config\n", "msg": "Creating physical volume '/dev/md125' failed", "rc": 5}
2022-05-13 23:19:56 ICT - {
"status" : "OK",
"msg" : "",
"data" : {
"uuid" : "ef58438e-c877-4304-a0f6-ab94fafd4b0c",
"counter" : 22,
"stdout" : "fatal: [xxx.xxx.xxx.xxx]: FAILED! => {\"changed\": false, \"err\": \" Cannot use /dev/md125: device is rejected by filter config\\n\", \"msg\": \"Creating physical volume '/dev/md125' failed\", \"rc\": 5}",
"start_line" : 20,
"end_line" : 21,
"runner_ident" : "83aa62d0-d2d8-11ec-b1bf-005056a0e9cd",
"event" : "runner_on_failed",
"pid" : 177889,
"created" : "2022-05-13T16:19:56.427041",
"parent_uuid" : "005056a0-e9cd-4faf-cce9-000000000011",
"event_data" : {
"playbook" : "create-brick.yml",
"playbook_uuid" : "a93bae64-f6fe-4293-aca6-407e3a1463bc",
"play" : "all",
"play_uuid" : "005056a0-e9cd-4faf-cce9-000000000008",
"play_pattern" : "all",
"task" : "Create volume groups",
"task_uuid" : "005056a0-e9cd-4faf-cce9-000000000011",
"task_action" : "lvg",
"task_args" : "",
"task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/gluster-brick-create/tasks/main.yml:36",
"role" : "gluster-brick-create",
"host" : "xxxx.xxx.xxx.xxx",
"remote_addr" : "xxx.xxx.xxx.xxx",
"res" : {
"msg" : "Creating physical volume '/dev/md125' failed",
"rc" : 5,
"err" : " Cannot use /dev/md125: device is rejected by filter config\n",
"invocation" : {
"module_args" : {
"state" : "present",
"vg" : "my_vg_brick1",
"pvs" : [ "/dev/md/VOL-KVM-Nx" ],
"pv_options" : "--dataalignment 256K",
"pesize" : "4",
"vg_options" : "",
"force" : false
}
},
"_ansible_no_log" : false,
"changed" : false
},
"start" : "2022-05-13T16:19:55.983964",
"end" : "2022-05-13T16:19:56.426739",
"duration" : 0.442775,
"ignore_errors" : null,
"event_loop" : null,
"uuid" : "ef58438e-c877-4304-a0f6-ab94fafd4b0c"
}
}
}

Changes

 N/A

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.