DBaaS

From the Command Line into the to the Oracle Cloud – the ORACLE-DBCS-CLI Utility

The Oracle ORACLE-DBCS-CLI utility is a command-line interface tool to manage the lifecycle of your Database as as Service instances in cloud.oracle.com. You can create and delete instances, verify and apply patches, scale your instances up and down and many more. The advantage is that you can work with configuration files. And I like to work with such files, if I need 10 instances with the same shape, I have only to search and replace the instance name – fire!

The ORACLE-DBCS-CLI utility is available for Unix platforms at the moment. I have installed ORACLE-DBCS-CLI on a Oracle Linux 6.5, JDK version 1.7 or higher is required.

Download and Documents

The software can be downloaded here: http://www.oracle.com/technetwork/database/database-cloud/public/downloads/index.html

The  ORACLE-DBCS-CLI documentation starts here: https://docs.oracle.com/cloud/latest/dbcs_dbaas/CSDBI/GUID-E9F2219D-EBDC-44B3-BE22-BEED6C2BAB74.htm#CSDBI-GUID-E9F2219D-EBDC-44B3-BE22-BEED6C2BAB74

Installation

After download the software has to be extracted on target server. My install directory is /u00/app/oracle/tools/dbcs-cli/ – take care about the release number.

What you get:

The dboplan.dat.tmpl is a template file which contains all required variables for working with cloud.oracle.com. For some actions you only need a subset of them.

Get Domain Information

To gather information about your created instances, you need this four parameters.

sm_url Endpoint URL – Attention: for EMEA it’s https://dbcs.emea.oraclecloud.com/paas/service/dbcs/api/v1.1/instances and not https://dbaas.oraclecloud.com/paas/service/dbcs/api/v1.1/instances as written in the documentation
user_name Your cloud.oracle.com user name
password Your cloud.oracle.com password
identity domain Your identity domain – e.g. trivadis4703

 

For this action, I have created a small configuration file which contains these four lines:

Now I can execute ORACLE-DBCS-CLI with the configuration file, all my instances in the specified domain are listed:

Get Patch Information

For this action you need your cloud SSH key and the host, IP does also work if the host name is not resolved by DNS, /etc/hosts etc.

Creating a new Instance

It is very easy to create a new instance, but here you need the full content from the configuration template. I have created my own template called cloud10.dat to create an instance with shape OC3 and no backup, a new ssh key will be created and uploaded by the ORACLE-DBCS-CLI. I have specified these parameters, all other I let them on default values:

sm_url Endpoint URL – Attention: for EMEA it’s https://dbcs.emea.oraclecloud.com/paas/service/dbcs/api/v1.1/instances and not https://dbaas.oraclecloud.com/paas/service/dbcs/api/v1.1/instances as written in the documentation
user_name Your cloud.oracle.com user name
password Your cloud.oracle.com password
identity domain Your identity domain – e.g. trivadis4703
vm_name Your new instance name – e.g. CLOUD10
vm_shape Your shape – e.g. OC3
db_sid Your SID – e.g. cloud10
db_dbname Your DB name – e.g. cloud10
db_passwd Your SYS password – it have to contain a numer, a # or a _ character
db_bkup_disk Backup = No

 

Executing ORACLE-DBCS-CLI with my Configuration File cloud04.dat

Unfortunately there is no logfile available, but If you have set a wrong shape or a too simple password, you will get an error.

That’s all folks – in cloud.oracle.com you can see now that your instance will be created immediately.

cloud04

My Template cloud04.dat

Summary

The ORACLE-DBCS-CLI Utility is a great tool to manage the lifecycle of your Oracle cloud instances in th CLI. With a small set of templates and configuration files, you are able to manage your cloud instances without any browser.

cloud.oracle.com – Create a Storage Container for a Backup in the Cloud

Why to use the Oracle Backup  Cloud storage?

During the instance creation process – the full deployed instance, not for the virtual machine – you can select if you want to store your database local or in the cloud – or both. The benefit to store a instance backup by RMAN in the cloud? Storage redundancy, easy access, full RMAN integration and with a service price of $33 / TB / Month it’s a real storage option.

Here is the step where you can select the backup storage option.

cloud_storage_1

Information Gathering

Before you can use this storage, you have to configure a storage container first. This can be done via REST command for example by a tool like curl. In this example I used curl for Windows. To create a storage container you have to know an authentification token (X-Auth-Token) and the URL (X-Storage-Url)  where the storage is presented. The storage name is composed by the word Storage and your Identity Domain name. For example my domain is called chtrivadis4703, the name of the storage user is Storage-chtrivadis4703.

The GET command:

There are some more lines with connection output text, I have removed then so it’s better readable:

Create a Storage Container

Now we know the X-Auth-Token and the X-Storage-Url, so we can create a new storage container called Backup.

For the verification I used Cloud Berry Explorer for OpenStack Storage (http://www.cloudberrylab.com/free-openstack-storage-explorer.aspx) – the container Backup was created successfully.

cloud_storage_2

Configure the Instance to use the Storage

Fill in the name of the cloud created storage container here and finalize the instance creation process.

cloud_storage_3

If the credentials or the storage container name is wrong, you will get an error message:

cloud_storage_4

Enjoy the new Cloud Storage

The new instance will be automatically configured to use this cloud storage as backup destination. But this container is not only for a backup of the cloud instances, you can use the Oracle Database Cloud Backup Module to backup your on-premise instances into the Oracle cloud. But this will be content in one of my next posts…

Links

Lost Disk after DBaaS Restart via Cloud Service Console

I have created a DBaaS instance three weeks ago, and now it was time to attach new storage in the Oracle Compute Cloud Service  to have additional disk space for some Trivadis tools. But after a reboot with a mouse-click in the DBaaS Cloud Service Console, the disk was lost and had to be re-attached.

The procedure to add additional storage is documented in this Oracle tutorial: http://www.oracle.com/webfolder/technetwork/tutorials/obe/cloud/dbaas/OU/IntroDBaaS/AddStorageDBaaSInstance/AddStorageDBaaSInstance.html

Creating additional  Storage in the Oracle Compute Cloud Service

In the Create Storage Volume menu, I have created a new storage volume of 10GB and attached it to my running DBaaS instance called TrivadisExPerf.

attach01

OS Verfication

After some minutes waiting, I logged in on the database server to verify the operation. The disk was attached as device xvdg.

Creating a Filesystem

Then I created a new partition with fdisk, formatted the partion as Ext4 and mounted it as /u05. At this point, I did not add something to /etc/fstab. But this had no impact for later actions.

Restart via DBaaS Cloud Service Console

The DBaaS restart was executed by the Service Console. In the background the DBaaS service was restarted automatically. The process is visible in the console and you can see the progress. I don’t know why Oracle requires more than 10 minutes for a simple restart, in some cases more than 20 minutes…

attach02

The Storage is lost…

After the restart I logged in again in the terminal. And what do I see? The storage was no more attached at the instance. There was no /dev/xdg or /dev/xdg1 visible.

After re-attaching the storage to the instance in the Oracle Compute Cloud Storage menu, the storage was back again. And I tried another way to restart a server…

attach03

Restart in the Terminal

First I checked again if the storage was attached. The device /dev/xdg was here.

Then I restarted the server with the shutdown command.

And after the restart, the storage was still here. The device /dev/xdg was available. And it didn’t require 10 mins, after 2 mins the server was back again and ready to log-in.

Summary

If you want to attach additional storage via Oracle Compute Cloud Service, it’s better at the moment NOT to restart your DBasS service instance via the console. Just use the good old shutdown command. I am sure, this can not be a feature, this is must be a bug.

Additional Information

When you attach a disk with the Scale Up function, then it works properly.

attach03

Now you have two ways how to attach additonal storage which are working. It’s like in the movie Indiana Jones and the Last Crusade:

You must choose. But choose wisely…

Patching a DBaaS Database in the Oracle Cloud

To apply a Patch on a Database which is hosted by Oracle is done with one mouse click. This service is available only if you have selected the full database service during the DBaaS order process.

To the manual: https://docs.oracle.com/cloud/latest/dbcs_dbaas/CSDBI/GUID-50BDEF7D-A30E-4B32-BAE7-486538413E2D.htm#CSDBI3422

Patchset Availability

As soon as a new Oracle patchset is available, you can see that in the database management dashboard of the selected database. The precheck function verifies if the selected database is ready to patch.

patch01

Precheck Result

After 2 until 3 minutes and a refresh of the page you see the result.

patch02

 

 

 

 

 

 

 

Command Line Interface dbpatchm

If you have no web access, all these steps can be executed manually in the shell. Therefore a command line tool called dbpatchm is available. You have to login as user root before using this tool.

List of available commands:

To the dbpatchm Documentation: https://docs.oracle.com/cloud/latest/dbcs_dbaas/CSDBI/GUID-50BDEF7D-A30E-4B32-BAE7-486538413E2D.htm#CSDBI3422

Patch Apply

If the prerequisite check is successful, we can apply the patch by selecting Patch.

patch03

If there are any Java Cloud Services (Weblogic Servers) which are using the selected database too, you can force to apply the patchset.

patch04

The patch progress is show in the dashboard.

patch05

In the background, Oracle transfers a new Oracle image where the selected PSU is already applied, extract it and changes the /u01/app location afterwards. During the patch apply process, the image is located under /u01/donwload:

Patch installed

After 45 minutes, the database is patched and up and running again.

patch07

The verification with OPatch shows the correct patch numbers. Here is an extract from the OPatch output. You can see that the apply-date is the 14th of July and that Oracle has installed bundle patch with patches for the RDBMS and the OJVM. I remember, in the background Oracle extracts a fully patched ORACLE_BASE / ORACLE_HOME directory for later usage.

The data dictionary is up to date too – the query on dba_registry_patch shows the correct PSU number:

Patching kills the EM12c Agent

Oracle is creating a new directory for the software, the old stuff which was in directory /u01/app is copied in /u01/app.ORG. So the EM12c Agent is not available anymore.

patch06

You have two possibilities to get the EM12c agent back:

  • Copy the installation directories from /u01/app.ORG back into /u01/app and re-attach the Agent Home in the inventory
  • Clean up the agent in EM12c and execute a reinstall

To avoid future problems, you can attach a new disk and use them for the agent or for other tools.

Summary

Patching a DBaaS database in the Oracle Cloud is very easy. All important steps are done automatically. In the background Oracle creates a new ORACLE_BASE directory in /u01. But you have to take care about installing your own tools or for example an EM12c agent. Also the database logs are no more available in the base directory. But you can find the in an subfolder of the direcory /u01/app.ORG where the old stuff is located.

Oracle Enterprise Manager 12cR5 – Clone a PDB to the Oracle Cloud

Cloning an on-premise PDB to the Oracle Cloud is basically done in three steps. 1. Select your PDB – 2. Start Clone Menu – 3. Verify new PDB. In the basic clone menu, Oracle uses some default settings in the background, for example the source datafiles will be packed in a compressed file which is located in /tmp directory of the source server. When your /tmp directory is too small, the clone process fails. Why this directory is used? I don’t know. But it could be a problem when you clone bigger databases. Maybe we can edit the clone procedure, but this is a task for me for a day when it’s raining outside… All cloning steps are well monitored, in case of an error you see a detailed error message so you can go on from the step where the error occurred.

To clone a PDB, there are some prerequisites which has to be fulfilled like same characterset, endianess and options. Don’t worry, if your source database has a wrong characterset, the clone procedure will be stopped.

The source PDB

My source PDB is a small database called CRM02 located in a Trivadis Datacenter – the container database is called TVD12CDB. I want to clone the local PDB to the Cloud database TVDHIPER.

cloud_2_cloud02

Start the Clone Procedure

In the menu Oracle DatabaseCloning select “Clone to Oracle Cloud”.

cloud_2_cloud02

Source and Destination Information

Enter the source and destination information like credentials, passwords etc. As destination Database Host Credentials select the SSH credential.This credebtial will be used to copy the files to the new server. In the right top corner you have the buttons Advanced and Clone. Clone starts the process with default settings immediately, if you select Advanced you can set destination directory, scheduler the execution etc. I select Clone.

cloud_2_cloud03

The Process starts immediately

This is the main menu where all the steps are monitored. You see all the details which are executed in the background. It’s an excellent overview.

cloud_2_cloud04

I one of the firsts steps, Oracle creates the PDB XML manifest and puts all datafiles in a compressed tar.gz file. As I said in one the first lines in this blog post, the directory is /tmp…

The tar.gz file will be transferred to the new target and after the transfer it’s deleted. But you have to clean up the datafiles manually.

What happens on Target Server

The tar.file will be extracted in a new folder and then the database will be plugged in. Here is the tar.gz file on target server. After the successful plug-in, the files will be deleted at this location. Oracle uses OMF – Oracle Managed Files, the extracted files are then located in this directory structure.

If the plug-in action fails, you can see that immediately in EM12c. During my clone process, an error occured when Oracle wants to apply a patch to the new PDB.

cloud_2_cloud05

In this case, the plugged in PDB has a wrong state. Oracle was not able to apply the datapatch because the pluggable database was not in UPGRADE state.

cloud_2_cloud_error01

As workaround I logged in as SYSDBA into the target database in the Oracle cloud and changed the mode manually.

Then I restarted the procedure from the position where the error occurred.

retry

Welcome to the cloned PDB

After some minutes, all steps are done, no errors. the pluggable database is now cloned and up and running. The task has status succeeded.

cloud_2_cloud06

EM12c Integration

EM12c target discovery is not required, the new pluggable database CRM02_CL will be added automatically as new target.

cloud_2_cloud07

Summary

First I wanted to try to clone a PDB from cloud database to cloud database, but this is not implemented at the moment. So I tried out to clone an on-premise database to the cloud. I have tested the basic functionality with a small database and it works fine. There are some issues like why does Oracle write files to /tmp or why has the plugged-in PDB the wrong state to apply the patches. But in general when you fulfill all the prerequisites to clone a PDB like character set etc. as you have to to on conventional way too, it runs. One of my next tests will be how it works when the source PDB version is not 12.1.0.2, when it’s a 12.1.0.1 PDB. But I assume it works as designed, there will be a message in PDB_PLUG_IN_VIOLATIONS view and the I have to solve it manually. Or do you think Oracle will upgrade my PDB automatically? We will see… 🙂