O3L 1.0: OperationalError: (OperationalError) (1040, 'Too many connections') None None

(Doc ID 2004717.1)

Last updated on NOVEMBER 23, 2017

Applies to:

Oracle OpenStack for Oracle Linux - Version OpenStack 1.0 to OpenStack 1.0 [Release ODOS1]
x86_64

Symptoms

Symptoms are variable when encounter this issue, e.g. operations failed on Horizon Web GUI, or command line failure "returned with HTTP 500".

Error messages in /var/log/nova/nova-api.log:

2015-04-28 11:24:21.023 20921 ERROR nova.api.openstack [req-37fe2617-5f33-4fb6-b151-2262089c7f14 d3320a73a99340948b5653ca75fb3c1d 9896928027c44d73a41b55d78bdaf58b] Caught error: (OperationalError) (1040, 'Too many connections') None None
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack Traceback (most recent call last):
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/__init__.py", line 125, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return req.get_response(self.application)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1296, in send
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     application, catch_exc_info=False)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/request.py", line 1260, in call_application
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return resp(environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py", line 684, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return self.app(env, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return resp(environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return resp(environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/routes/middleware.py", line 131, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     response = self.app(environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 144, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return resp(environ, start_response)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 130, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/webob/dec.py", line 195, in call_func
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 917, in __call__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     content_type, body, accept)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 983, in _process_stack
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py", line 1070, in dispatch
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return method(req=request, **action_args)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/api/openstack/compute/servers.py", line 933, in create
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     read_deleted="no")
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/compute/flavors.py", line 253, in get_flavor_by_flavor_id
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return db.flavor_get_by_flavor_id(ctxt, flavorid, read_deleted)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/db/api.py", line 1450, in flavor_get_by_flavor_id
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return IMPL.flavor_get_by_flavor_id(context, id, read_deleted)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 164, in wrapper
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return f(*args, **kwargs)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 4376, in flavor_get_by_flavor_id
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     order_by(asc("deleted"), asc("id")).\
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/query.py", line 2282, in first
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     ret = list(self[0:1])
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/query.py", line 2149, in __getitem__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return list(res)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/query.py", line 2353, in __iter__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return self._execute_and_instances(context)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/query.py", line 2366, in _execute_and_instances
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     close_with_result=True)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/query.py", line 2357, in _connection_from_session
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     **kw)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 799, in connection
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     close_with_result=close_with_result)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 805, in _connection_for_bind
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return engine.contextual_connect(**kwargs)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 1661, in contextual_connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     self.pool.connect(),
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 272, in connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return _ConnectionFairy(self).checkout()
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 433, in __init__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     conn = self.connection = self._connection_record.get_connection()
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 346, in get_connection
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     self.connection = self.__connect()
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py", line 379, in __connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     connection = self.__pool._creator()
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return dialect.connect(*cargs, **cparams)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/default.py", line 283, in connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return self.dbapi.connect(*cargs, **cparams)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/MySQLdb/__init__.py", line 81, in Connect
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     return Connection(*args, **kwargs)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack   File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 187, in __init__
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack     super(Connection, self).__init__(*args, **kwargs2)
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack OperationalError: (OperationalError) (1040, 'Too many connections') None None
2015-04-28 11:24:21.023 20921 TRACE nova.api.openstack
2015-04-28 11:24:21.025 20921 INFO nova.api.openstack [req-37fe2617-5f33-4fb6-b151-2262089c7f14 d3320a73a99340948b5653ca75fb3c1d 9896928027c44d73a41b55d78bdaf58b] http://10.10.10.10:8774/v2/9896928027c44d73a41b55d78bdaf58b/servers returned with HTTP 500

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms