DBaaS Creation Fails on Oracle Cloud At Customer Classic Plane
(Doc ID 2493806.1)
Last updated on OCTOBER 04, 2021
Applies to:
Oracle Database Cloud Service - Version N/A and laterOracle Cloud at Customer - Version N/A and later
Information in this document applies to any platform.
This KM note is applicable for Oracle Cloud at Customer Database Cloud Service Only
Symptoms
Customer unable to create Oracle Database Cloud Service on Oracle Cloud At Customer Environment.
The Service creation fails with Timeout after 116 mins and the Compute resources get deleted post failure.
However interesting note is that the same DBCS get successfully created if Customer selects Service Creation Without Backups Configured.
From the UI:
Dec 17, 2018 8:19:47 PM UTC Failed to establish SSH connectivity to VM(s) after [116] minutes...
Dec 17, 2018 8:19:50 PM UTC Initiating Database Service Termination...
Dec 17, 2018 8:19:50 PM UTC Skipping delete backups for Database Service instance [UFIS] since the request parameter for delete backups is not set.
Dec 17, 2018 8:22:29 PM UTC Database Service record removal complete...
Dec 17, 2018 8:23:10 PM UTC Stopped all Compute resources...
Dec 17, 2018 8:23:13 PM UTC Removed Compute resources...
Dec 17, 2018 8:23:19 PM UTC Activity Ended
From PSM Logs:
{panel:title=PSM failed Job}
k-dbaas-service,myemail@oracle.com,2018-12-21T09:45:13.865+0000,2018-12-21T09
:45:13.865+0000,2018-12-21T09:59:47.092+0000,2018-12-21T09:59:47.091+0000,Fail
ed
115547,DBaaS,NBEONTRC,patch-service-no-maintenance-workflow,myemail@oracle.com
,2018-12-21T10:17:47.368+0000,2018-12-21T10:17:47.368+0000,2018-12-21T10:32:1
8.919+0000,2018-12-21T10:32:18.918+0000,Failed
115816,dbaas,UFISC,create-dbaas-service,myemail@oracle.com,2018-12-24T08:13:5
5.922+0000,2018-12-24T10:18:32.606+0000,2018-12-24T12:22:59.004+0000,2018-12-2
4T12:22:58.997+0000,Failed
115844,DBaaS,NBEONTRC,patch-service-no-maintenance-workflow,myemail@oracle.com,
2018-12-24T12:32:35.413+0000,2018-12-24T12:32:35.413+0000,2018-12-24T12:47:2
0.066+0000,2018-12-24T12:47:20.065+0000,Failed
116089,dbaas,UFISC,create-dbaas-service,myemail@oracle.com,2018-12-27T06:46:2
9.214+0000,2018-12-27T08:51:13.929+0000,2018-12-27T10:56:58.247+0000,2018-12-2
7T10:56:58.239+0000,Failed
{panel}
On further checking the Failed PSM Job Log:
Checking PSM job log, noticed that between these two steps, there was a very
long delay (almost 2 hours):
2019-01-04 19:20:39 - Output from cmd
/var/opt/oracle/ocde/assistants/bkup/bkup -out
/var/opt/oracle/ocde/res/bkup.out -sid="UFISDR" -libopc_mode="prod"
-bkup_cron_entry="yes" -bkup_use_rcat="no"
-hostname="UFIS18WB.compute-9999999.gr2.internal"
-oracle_home="/u01/app/oracle/product/18.0.0/dbhome_1"
-bkup_oss_recovery_window="30" -dbname="UFISDR" -service_name="DBCS_DEFAULT"
-dbtype="sifs" -bkup_cfg_recovery_window="30" -bkup_cfg_files="yes"
-edition="enterprise" -bkup_nfs="no"
-bkup_oss_url="https://a999999.gr2.ocm.s7130781.oraclecloudatcustomer.com/v1/Storage-a999999/dbcs-UFIS18WB" -bkup_oss_user="myemail@oracle.com"
-bkup_nfs_recovery_window="30" -version="18000" -archlog="yes"
-oracle_base="/u01/app/oracle" -firstrun="yes" -action="config"
-bkup_oss="yes" -bkup_disk_recovery_window="7" -bkup_disk="no" -action=config
run on localhost is:
2019-01-04 21:05:50 - Starting BKUP
.
.
.
.
And, in the inner detail, this file is tried to download 10 times with no
success:
Downloading Oracle Database Cloud Backup Module Software Library from file
opc_linux64.zip.
Operation timeout, retrying...
ERROR : OPC installer has timed out after 10 attempts.
WARN : OPC installer has failed after 10 attempts.
WARN : Use local copy instead.
Changes
No Changes were done in particular... The DBCS Creation is using a Image that has Old DBCS ToolSet (Jan 2018) instead of latest one.
For most of Oracle Cloud At Customers Staging Environment (Including DBCS and EXACS), the Images will not have latest Tooling. Instead the expectation is that the Customers will have to manually upgrade the Tooling after the Service is created successfully.
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |
| References |