Author Archive: Martin Berger

Monitor your Oracle Cloud Infrastructure Autonomous Database with your On-Prem Oracle Enterprise Manager 13c

This blog post describes how to use an on-prem Oracle Enterprise Manager 13c to monitor Oracle Cloud Infrastructure Autonomous Databases without internet traffic. In this case, a VPN connection from the data center Mohnweg/Jurasüdfuss/Switzerland to the OCI data center in Zurich is up and running. This method works with a Fast Connect too.

From the Oracle Enterprise Manager 13.5 Administrator Guide:

Oracle Enterprise Manager supports the following Autonomous Databases and the term “Autonomous Databases” in this guide collectively refers to them:

  • Autonomous Data Warehouse – Dedicated
  • Autonomous Transaction Processing – Dedicated
  • Autonomous Data Warehouse – Shared
  • Autonomous Transaction Processing – Shared

Architecture

  • On-Prem: Oracle Enterprise Manager 13.5 RU2 – OL 7
  • Oracle Cloud Infrastructure: Autonomous Transaction Processing – Shared

Requirements

  • VPN connection up and running
  • EM 13c up and running
  • Routing and security list from on-prem to OCI and vice-versa in place
  • An OCI Network Security Group NSG created – required for the ADB private endpoint connection

Network Security Group NSG

Allow ingress traffic for port 443 (Oracle Database Actions) and 1522 (Oracle Net).

Create and Autonomous Transaction Processing Database

Choose workload type, version, OCPU count and license type. Configure the section network access for private access only. Take care, the optional Host name prefix is later used for the connection as hostname. Choose it wisely. With a private endpoint, all network traffic moves in the private subnet within a VCN. It keeps all traffic to and from your Autonomous Database off of the public internet.

ATP – Database Actions – Access

In the ATP page of the new created Autonomous Database, a click on Database Access shows you the URL to the web console. This URL cannot be resolved from your local workstation, the hostname is unknown. The hostname has to be resolved first by your DNS (quick and dirty: local hosts file) or you use the IP address instead which is provided in the Autonomous Database Details page.

The private endpoint IP 192.168.201.11 is in our OCI private subnet. I have added it temporarily to my local workstation hosts file.

ATP – Database Actions – User ADBSNMP

Login into Database Actions with the provided URL to enable the already existing user ADBSNMP for monitoring.

  • User: ADMIN
  • Password: Your ADB password

In the Database Actions Launchpad, select DATABASE USERS.

Edit user ADBSNMP, unlock the account and set a password. This password is later used in Oracle Enterprise Manager 13c.

ATP – Download Wallet

On the Autonomous Database Details page, download the DB Connection wallet. The wallet and the wallet password are later used in Oracle Enterprise Manager 13c.

Oracle Enterprise Manager 13c – ATP Host Name Resolution

Oracle uses FQDN in the ADB wallet tnsnames.ora file. Verify if the hostname which is listed above on the details page as Private Endpoint can be resolved by the on-prem OEM host. Alternative: Add this line to the /etc/hosts file. Attention: this is not the same domain as used for the Database Actions console (oraclecloudapps.com vs. oraclecloud.com). Tip for troubleshooting: Test the database connection with your local SQL Developer and user ADBSNMP, for a local SQL*Plus connect you have to extract the wallet file and configure Oracle Net.

Connectivity check firewall and routing with Telnet:

Oracle Enterprise Manager 13c – Add Targets Manually

Login as SYSMAN – SetupAdd TargetAdd Targets Manually.

Add Non-Host Targets Manually.

As Agent Host, you can choose your local agent which is running on the OEM host. Select target type for Autonomus Data Warehouse or Autonomus Transaction Processing.

  • Set Target Name, choose the OCI Client Credential Wallet and set Monitoring Password.
  • Test the connection.
  • Next and Submit.

The target will be added in the next minutes.

Oracle Enterprise Manager 13c – ADB Detail Page

The Autonomous Database is added as new database target and can be monitored on the same way as the on-prem databases. As it is a service, the metrics are limited. For example: when the ADB is stopped, the Oracle Enterprise Manager shows an Availability Evaluation Error.

Summary

Adding an Oracle Cloud Infrastructure Autonomous Database in an on-prem Oracle Enterprise Manager EM13c by using a private endpoint is a great thing. Take care about the routing and the firewall rules, verify all connections before you add an ADB as a new target. Even when the metrics are limited, for a basic monitoring (+) is it ok.

Next step: I want to deploy an  EM13c agent on a DBCS system. The MOS note DBCS: How to Deploy EM Agent on Cloud / DBCS Instance (Doc ID 2400965.1) shows the old OCI classic way, a SR to ask if this is allowed in OCI Gen. 2 too is already raised.

This was the last blog post of the year 2021 – see you in 2022! #happynewyear

Oracle Enterprise Manager 13c Release 5 – Time for Release Update 2

Since a few days the 13.5.0.2 Enterprise Manager Cloud Control Base Platform Monthly Release Update (RU) 2 is available to apply. Time to update my 13.5 lab environment at home to get a first experience about a lot of new features like Oracle Cloud Infrastructure Resource Discovery, Oracle Enterprise Manager Dashboards, Database Scheduler Jobs Metrics and many more.

Notes and Links

My Oracle Support Notes

  • 13.5.0 Enterprise Manager Cloud Control Base Platform Monthly Release Update (RU) 2 (Doc ID 2822316.1)
  • Enterprise Manager 13.5 Main Release Update List (Includes Plug-ins) (Doc ID 2760230.2)

Other Links

Patch Number 33456001

The Environment

  • Oracle Enterprise Manager 13.5 running on Oracle Linux 7.9
  • Oracle Enterprise Edition Repository 19.13 Single Tenant
  • Oracle Restart / ASM 21.0.0
  • Additional Oracle Linux Server with 19.3.0 Container Databases
  • All targets up and running
  • My Oracle Support connected
  • Patch software staged directory in /u01/app/oracle/stage/33456001

Installed Plugins

Verification by emcli. Plugins actually have release number 13.5.1.0.0.

 

Prerequisites

For the readme:

  1. On the Repository Database, apply Database Release Update 19.11.0.0.0 Patch 32545013 and Overlay patch for 17777718 on top of Database Release Update 19.11.0.0.0. Or apply Database Release Update 19.12.0.0.0 Patch 32904851 or its later Database Release Update version patch.

  2. On Windows, apply Oracle Repository Creation Utility patch 33053642 on all OMSes.

  3. Ensure that you have the latest version of OMSPatcher version 13.9.5.1.0 for Enterprise Manager 13.5.0.0 release on all OMSes.

In my case I had to update OMSPatcher and his little sister OPatch. For upgrading these components, I recommend these notes:

  • 13.5: How To Upgrade Enterprise Manager 13.5 Cloud Control OMSPatcher Utility to Version 13.9.5.1.0 (Doc ID 2809842.1)
  • Patch 28186730: OPATCH 13.9.4.2.7 FOR EM 13.4, 13.5 AND FMW/WLS 12.2.1.3.0, 12.2.1.4.0 AND 14.1.1.0.0

OMSPatcher

OPatch

Software Directory

The release update is transferred to the local stage directory:

Pre-Check

Run the pre-check to analyze the existing Oracle Enterprise Manager system.

These warning messages about patches which can not applied can be ignored, I don’t have installed these plugins like SMF (Enterprise Manager for Fusion Applications):

Apply

1st – Stop the Oracle Management Server

2nd – Apply RU with OMSPatcher

As in some blog posts before, I don’t use a credential file, therefore the username (if not using weblogic which is default) and the password for the Weblogic AdminServer has to be provided. The job runs some minutes. The OMS will be started automatically.

Verification

Get the Release Update Information, the inventory was updated:

Next Step

  • Plugin Upgrade
  • Agent Patch

Summary

Some weeks ago there was a presentation with all these new features like customizable dashboards. The update process himself was easy and straight forward as we knew from other releases. Finally the Release Update 2 is arrived. Let’s see in the next days what’s in the box.

Oracle Cloud Infrastructure Resource Naming Conventions – A short Friday Blog Post

Cleaning up the OCI Resource Chaos

This week I have removed all my Oracle Cloud Infrastructure lab resources and cleaned up my compartment to start from scratch building environments with the Automation Manager. There were a lot of resources with names like webserver01, vcn-lab-01, block-volume-web-clone-47 and so on. When I have realized this naming chaos, I have decided to spend some minutes today Friday to think about a small naming convention for my future Oracle Cloud Infrastructure projects.

Why having a naming convention? Wikipedia says about Naming Convention:

In computer programming, a naming convention is a set of rules for choosing the character sequence to be used for identifiers which denote variables, types, functions, and other entities in source code and documentation.

Sounds good, this is what I need. There are some proposal docs in the web about naming convention from Azure and TrendMicro – I used them as an inspiration.

Naming Convention

To identify an OCI resource, tagging is not enough. I like to see it at a glance what a resource is doing, which project and, if required, some additional information. This here is just a idea how it could work. Any other ideas are welcome :-). I have decided to define a resource by:

  • A resource type like bv for Block Volumes
  • A region key – based on https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm
  • An environment code – like letter t for test, d for development and p for production
  • A stack – this could be a project name or a purpose
  • A instance – the number of the resource if more than one is created

And in some cases where this information is not enough:

  • An additional –  like read-only, protected etc.

Examples

Resource Name Purpose
vcn-zrh-t-newcrm-001 VCN, region Zurich, test, NEWCRM project
sn-prv-fra-p-transfer-001 Private Subnet, region Frankfurt, production, TRANSFER project
nsg-zrh-d-ipsec-001 Network Security Group, region Zurich, development, IPSEC project
bv-zrh-t-ipsec-olvm-engine-001 Block Volume, region Zurich, test, IPSEC project, compute instance olvm-engine
log-ams-p-crm-read-001 Log Group ,region Amsterdam, production, CRM project, read only
ci-fra-t-science-ngnix-003 Compute Instance , region Marseille, test, SCIENCE project, NGINX server

 

A first View – Network Visualizer

This looks not so bad.

Summary

If you want to bring an order into your naming chaos, then you have to define a naming convention. There are for sure many other ideas around the world. How do you handle it? Just drop me a message by LinkedIn or Twitter and let me knolw if I can add your idea here in this blog too.

Oracle 11g – let’s move that old Stuff to the Oracle Cloud

This blog post describes the lift and shift of an on-prem Oracle 11g Enterprise Edition to Oracle Cloud Infrastructure by using Oracle RMAN paired with OCI Object Storage. Works for other versions > Oracle 11g / Enterprise Edition too (the tablespace encryption method may change).

Architecture

Q&A

What’s the motivation of the lift and shift of a good old on-prem Oracle 11g database to Oracle Database Service?

  • Hardware runs out of lifecycle
  • Out scoping of an on-prem database which is only used for several times to compare old data
  • Regulatory, for example we have the keep and access the data for 10 yrs
  • Changing from on-prem licenses to the license included model
  • Upgrade tests for 19c and convert to Multitenancy Architecture
  • Part of company’s cloud strategy

Why using the OCI database service instead of a cheaper compute instance?

Only the database service allows tablespace encryption without any additional costs in the license included model. From my point of view, this is a must when running Oracle databases outside of the on-prem datacenter in any cloud. And for this case, 11g to 11g, it fit’s best without any configuration overhead. And finally, the root OS access gives me flexibility.

Are the other methods available than using RMAN backup/restore/recovery with the Oracle Database Backup Service?

Sure, according the Oracle docs:

  • Golden Gate
  • Data Transfer Service
  • Oracle Data Pump
  • Database Migration Service
  • RMAN Transportable Tablespaces

Why I like the Oracle Database Backup Cloud Service?

  • The on-prem RMAN backups have to be encrypted, no encryption, no Database Backup Service.
  • The configuration of the Backup Service Module is not really complicated, well documented and can be used 1:1 in on-prem and in the cloud. For 11.2.0.4,I need to install on-prem a patch to encrypt the backups (18339044 – RMAN-06770: backup encryption requires Enterprise Edition).
  • The backup configuration can be done in advanced without any pressure, the target database can be recovered at any time. This reduces once the restore is done the downtime when the final move should be done. Backing up an Oracle database by RMAN locally and to the cloud can co-exist without any problems.
  • The backup is stored in the Object Storage. With a replication policy, I can rebuild a new database in another region in an easy way.
  • Oracle Database Backup Module allows proxy configurations to backup to the cloud.

Links and My Oracle Support Notes

Migration Steps

  1. Create OCI Object Storage
  2. Configure on-prem database to use the Oracle Database Backup Service
  3. Create a new OCI Database Virtual Machine
  4. Configure OCI cloud database to use the Oracle Database Backup Service
  5. Clean up OCI cloud database
  6. Restore OCI cloud database from Object Store
  7. Encrypt tablespace

1. OCI Object Storage

A Object storage bucked called onprem-bucket is created in region eu-zurich-1:

 

2. Configure on-prem database to use the Oracle Database Backup Service

The installation of the Backup Service is described in this link here, you can download the OCI Backup Service Module: www.oracle.com/technetwork/database/availability/oracle-cloud-backup-2162729.html

On-Prem Database – Specifications

  • 11.2.0.4 Enterprise Edition
  • Database name DB11
  • Oracle Linux 7.9
  • About 60GB size
  • Oracle Backup Service to Object Storage enabled and configured
  • Regular RMAN inc0, inc1c and arc backups to the Cloud
  • Enabled RMAN compression and encryption (for the usage of the Backup Service the license is included)
  • Backup encrypted by password

On-Prem Database – Tablespaces and Datafiles

On-Prem Database – RMAN Backup to Oracle Cloud Infrastructure

Example RMAN Output where the media is the Oracle Cloud Infrastructure Object Storage in Zurich / Switzerland.

 

3. Create a new OCI Database Virtual Machine

A new 11g Oracle Database virtual machine is created, license included. 11g is not available for Logical Volumes unfortunately, therefore we need to use Grid Infrastructure.

Cloud Database – Specifications

  • 11.2.0.4 Enterprise Edition Database Virtual Machine
  • Oracle Linux 7.9
  • Grid Infrastructure (11g is not available for Logical Volumes unfortunately)
  • Database Name is DB11
  • Connected from the on-prem data center a) by VPN or b) by SSH-Tunnel via Bastion Host

Cloud Database – Controlfile, Tablespaces and Datafiles

Login as OS user opc, sudo to oracle and set environment.

Login as SYSDBA.

Query controlfile information.

Query datafile information. The existing datafiles will be dropped later before the restore starts.

Query tablespace information for encryption, the USERS tablespace is already encrypted. Encryption of SYSTEM, SYSAUX etc. is introduced in a later Oracle version and not available in 11g.

4. Configure OCI Cloud Database to use the Oracle Database Backup Service

Note: When the database is located in a VCN private subnet, to install and using the Oracle Database Backup Service module you have to configure these two resources:

  • Service Gateway for Object Storage access
  • NAT Gateway used by the Database Backup Service Module Library installation routine

For the installation the SSH private key from the existing on-prem Database Backup Service configuration is required and has to be transferred to cloud server. The fingerprint of the public ssh key is required.

A list of object storage endpoints is avaliable here: Object Storage Service API | Oracle Cloud Infrastructure API Reference and Endpoints

Cloud Database – Oracle Database Backup Service Installation and Configuration

Login to cloud virtual machine and sudo to OS user oracle

Create directory for the Database Backup Module installation, transfer and extract it.

Create directories for wallet and library.

Transfer the SSH key from on-prem to the cloud virtual machine into oci_wallet directory.

Install Oracle Backup Service Module, use the private key and public fingerprint from the on-prem installation. Set environment and start the installer. Use the bucket name from OCI Object Storage bucket which you have created first.

Installer Output.

A new parameter file is created which contains the OCI Object Storage information. This configuration file is used later in RMAN.

5. Clean up OCI Cloud Database and restart NOMOUNT

Login to cloud database virtual machine as OS user grid.

Set environment to database DB11.

Shutdown the database instance by using the database unique name.

Error message when environment for +ASM is set:

ASM Cleanup with environment +ASM – you have to add ORACLE_HOME manually

Login in ASM and remove existing controlfiles, datafiles, tempfiles and directories.

Set environment to DB11.

Start database NOMOUNT.

6. Restore OCI cloud database from Object Store

Restore and recovery of the database, the database has same name as the on-prem database. The datafile is migrated from file system to ASM. Required information for restore and recovery:

  • On-prem database DBID
  • Encryption password

Cloud Database – Login in RMAN and set Decryption Password set Source DBID

Cloud Database – Restore Controlfile from Object Storage

Use the library path and the path to the configuration file (OPC_FILE) properly.

Cloud Database – Mount Instance

Cloud Database – Restore Instance

Allocate channel for maintenance first.

Start restore.

Recover database – ignore the last line of the incomplete recovery.

Cloud Database – Open RESETLOGS

7. Encrypt Tablespace

Login as OS user oracle / SYS as SYSDBA to verify the existing situation.

Cloud Database – Verify Tablespaces

Cloud Database – Tablespace Encryption

We use the existing wallet and add a new TDE master key to the configuration. Show parameter for tablespace encryption.

Verify existing actual encryption situation, no tablespaces are encrypted.

Take user tablespaces offline – a small syntax provider script.

As SYS AS SYSDBA, set new Master Key

Encrypt tablespaces – a small syntax provider script.

Take encrypted tablespaces online, the encryption starts and the taking online action needs some time (depends on CPU and I/O).

A few minutes later, the user tablespaces are shown as encrypted.

Verify encrypted tablespaces by DBVerify – as you can see here, Total Pages Encrypted is shown.

Summary and what’s next

Migration of an on-prem database to Oracle Cloud Infrastructire by RMAN and Object Storage is a very nice method to bring not only older databases into the Oracle Cloud Infrastructure. Once there, you can leverage of OCI features like Data Safe, monitoring, backup to Object Storage and many more.

#ilikeit

Never stop Learning – why I love Oracle LiveLabs

Since over one and a half year, this week I was back in an onsite training, live people, live teaching. With a motivated junior DBA class, we started with all about Oracle architecture based on our Trivadis training O-AI – Oracle Architecture and Internals. The training is a mix between slides, demos and labs. Therefore during the course we run the training environments in Oracle Cloud Infrastructure, build by Terraform (Credits to Stefan Oehrli from oradba.ch which has ramped up the whole stuff). After the course at the end of the month, the environments will be cleaned up. And what’s next?

Training Environments

There are a lot of possibilities to get a deeper knowledge of all this Oracle stuff like processes, data encryption, multitenancy, datapump and so on:

But my actual favorite is Oracle LiveLabs!

Oracle LiveLabs

This platform is not only for DBAs, it has a lot of workshops for Application Developers, Data Scientists and DevOps Engineers too. There are different workshop types available:

  • get your free Oracle Cloud Infrastructure training environment for free during a time period like Oracle Database 19c New Features – run on LiveLabs
  • workshops which are running in a free tenancy
  • workshops what you can do in your own paid tenancy

At the moment there are 21 workshops where you get a live environment with all components you need virtual machines or database in Oracle Cloud Infrastructure like Oracle Multitenant Fundamentals, Database 19c – Automatic Indexing, 21c New Features on Autonomous Database and many more. All workshops are very well described, from the access to the initial setup and finally for the workshop himself too.

In this case I have decided to start the Oracle Multitenant lab to gather more information how PDB Snapshot Copy works.

1st – Search your training in the available workshops  and press Launch

2nd – Define where the Workshop should run

In this case, I want to reserve an environment. This is not possible for all workshops, you can see that in the workshop details if it’s possible to the an Oracle Cloud Infrastructure setup.

3rd – Define the Start Date and propose your SSH Public Key

With the key, you can get access to the training servers by SSH. In this case I want to start the workshop immediately. Otherwise define a start and end date. If you don’t want to start now, you will get a confirmation that the workshop is reserved and an email at the day where the workshop starts with the credential information.

4rd – View my Reservation

After some minutes, the status for the workshop is updated. As you can see here, in about three minutes from now, the environment should be ready. You will receive a confirmation mail.

5th – Launch Workshop

When the workshop is ready, the workshop can be launched.

6th – Workshop Details

All information you need is in the details like:

  • User name
  • Initial password for OCI
  • Compartment
  • Instance public IP

Here you have also the chance to extend your workshop reservation time. Follow the Get Started instructions to the bottom and push the button to move on the introduction. Step by step you are guide through the login and setup process. All labs contain a manual how to connect and to do the initial setup like starting listeners or get scripts from the OCI Object Storage.

There are the connection options how you can interact with the LiveLab:

  1. Connect using Cloud Shell
  2. Connect using MAC or a Windows CYGWIN Emulator
  3. Connect using Putty

Example code for the multitenancy lab preparation:

Summary

Oracle LiveLabs is another great opportunity to learn and train new stuff. All you have to take care now is to follow the workshop instructions and take care about the limited time. Enjoy it, learn new stuff and have fun! Oracle LiveLabs are easy to join, easy to set up and well described. This is why I love it 🙂