O3L 1.0: neutron.agent.dhcp_agent Stderr: ndnsmasq: failed to set SO_REUSE{ADDR|PORT} on DHCP socket: Protocol not available

(Doc ID 2001580.1)

Last updated on NOVEMBER 23, 2017

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 1.0 to OpenStack 1.0 [Release ODOS1]
x86_64

Symptoms

1. An IP Address 10.10.10.11 could be observed as allocated to the instance:

[root@o3l ~(keystone_admin)]# nova show d65cffaa-0f94-49cc-902f-b4d39171292b
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig                    | AUTO                                                     |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| OS-EXT-SRV-ATTR:host                 | o3l.cn.oracle.com                                  |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | o3l.cn.oracle.com                                  |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000008                                        |
| OS-EXT-STS:power_state               | 0                                                        |
| OS-EXT-STS:task_state                | spawning                                                 |
| OS-EXT-STS:vm_state                  | building                                                 |
| OS-SRV-USG:launched_at               | -                                                        |
| OS-SRV-USG:terminated_at             | -                                                        |
| accessIPv4                           |                                                          |
| accessIPv6                           |                                                          |
| config_drive                         |                                                          |
| created                              | 2015-04-21T08:06:11Z                                     |
| flavor                               | m1.tiny (afc5c749-eb79-4d6e-ba64-037e82d1e9bc)           |
| hostId                               | 2a27db05a3974bcedf978e5cab7758b16ab7112dce452c270a4d20ad |
| id                                   | d65cffaa-0f94-49cc-902f-b4d39171292b                     |
| image                                | ol6u5 (5d6f236b-e83d-407b-9366-4f0bad063eda)             |
| key_name                             | -                                                        |
| metadata                             | {}                                                       |
| name                                 | ol6u5-1                                                  |
| net1 network                         | 10.10.10.11                                              |
| os-extended-volumes:volumes_attached | []                                                       |
| progress                             | 0                                                        |
| security_groups                      | default                                                  |
| status                               | BUILD                                                    |
| tenant_id                            | 5e25a2c7f35043cda62fcbdb81535568                         |
| updated                              | 2015-04-21T08:06:15Z                                     |
| user_id                              | 670159e92c9a4385823dc1efd913ed3a                         |
+--------------------------------------+----------------------------------------------------------+


2. The instance IP Address is not reachable via the dhcp namespace ICMP echo requests:

[root@o3l ~(keystone_admin)]# ip netns exec qdhcp-566ad586-ea0e-436f-83fb-a5f2af0ab743 ping 10.10.10.11
PING 10.10.10.11 (10.10.10.11) 56(84) bytes of data.
From 10.10.10.10 icmp_seq=1 Destination Host Unreachable
From 10.10.10.10 icmp_seq=2 Destination Host Unreachable
From 10.10.10.10 icmp_seq=3 Destination Host Unreachable
^C
--- 10.10.10.11 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3955ms


3. Within instance, IP Address could not be obtained via DHCP after several network service restarts.

4. Errors could be seen in /var/log/neutron/dhcp-agent.log:

2015-04-21 02:32:21.215 20266 ERROR neutron.agent.dhcp_agent [-] Unable to enable dhcp for 566ad586-ea0e-436f-83fb-a5f2af0ab743.
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent Traceback (most recent call last):
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/dhcp_agent.py", line 148, in call_driver
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     getattr(driver, action)(**action_kwargs)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 182, in enable
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     self.spawn_process()
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 398, in spawn_process
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     specific_pid_file=pid_filename)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/external_process.py", line 178, in enable
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     process_manager.enable(reload_cfg=reload_cfg)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/external_process.py", line 83, in enable
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     ip_wrapper.netns.execute(cmd, addl_env=self.cmd_addl_env)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 467, in execute
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     check_exit_code=check_exit_code)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 77, in execute
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent     raise RuntimeError(m)
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent RuntimeError:
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-566ad586-ea0e-436f-83fb-a5f2af0ab743', 'env', 'NEUTRON_NETWORK_ID=566ad586-ea0e-436f-83fb-a5f2af0ab743', 'dnsmasq', '--no-hosts', '--no-resolv', '--strict-order', '--bind-interfaces', '--interface=tap02b947f8-d7', '--except-interface=lo', '--pid-file=/var/lib/neutron/dhcp/566ad586-ea0e-436f-83fb-a5f2af0ab743/pid', '--dhcp-hostsfile=/var/lib/neutron/dhcp/566ad586-ea0e-436f-83fb-a5f2af0ab743/host', '--addn-hosts=/var/lib/neutron/dhcp/566ad586-ea0e-436f-83fb-a5f2af0ab743/addn_hosts', '--dhcp-optsfile=/var/lib/neutron/dhcp/566ad586-ea0e-436f-83fb-a5f2af0ab743/opts', '--leasefile-ro', '--dhcp-range=tag0,10.10.10.0,static,86400s', '--dhcp-lease-max=256', '--conf-file=', '--domain=openstacklocal']
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent Exit code: 2
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent Stdout: ''
2015-04-21 02:32:21.215 20266 TRACE neutron.agent.dhcp_agent Stderr: '\ndnsmasq: failed to set SO_REUSE{ADDR|PORT} on DHCP socket: Protocol not available\n'

 

5. A dnsmasq process should be seen for the neutron dhcp service, however not:

[root@o3l images(keystone_admin)]# ps -ef | grep dnsmasq
root     21328  2960  0 02:36 pts/0    00:00:00 grep dnsmasq

 

6. dnsmasq version 2.48-14.el6.x86_64 is installed:

[root@o3l images(keystone_admin)]# rpm -qa | grep dnsmasq
dnsmasq-2.48-14.el6.x86_64
dnsmasq-utils-2.48-14.el6.x86_64 

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms