Uncategorized

Monitor your Oracle Cloud Infrastructure Autonomous Database with your On-Prem Oracle Enterprise Manager 13c

This blog post describes how to use an on-prem Oracle Enterprise Manager 13c to monitor Oracle Cloud Infrastructure Autonomous Databases without internet traffic. In this case, a VPN connection from the data center Mohnweg/Jurasüdfuss/Switzerland to the OCI data center in Zurich is up and running. This method works with a Fast Connect too.

From the Oracle Enterprise Manager 13.5 Administrator Guide:

Oracle Enterprise Manager supports the following Autonomous Databases and the term “Autonomous Databases” in this guide collectively refers to them:

  • Autonomous Data Warehouse – Dedicated
  • Autonomous Transaction Processing – Dedicated
  • Autonomous Data Warehouse – Shared
  • Autonomous Transaction Processing – Shared

Architecture

  • On-Prem: Oracle Enterprise Manager 13.5 RU2 – OL 7
  • Oracle Cloud Infrastructure: Autonomous Transaction Processing – Shared

Requirements

  • VPN connection up and running
  • EM 13c up and running
  • Routing and security list from on-prem to OCI and vice-versa in place
  • An OCI Network Security Group NSG created – required for the ADB private endpoint connection

Network Security Group NSG

Allow ingress traffic for port 443 (Oracle Database Actions) and 1522 (Oracle Net).

Create and Autonomous Transaction Processing Database

Choose workload type, version, OCPU count and license type. Configure the section network access for private access only. Take care, the optional Host name prefix is later used for the connection as hostname. Choose it wisely. With a private endpoint, all network traffic moves in the private subnet within a VCN. It keeps all traffic to and from your Autonomous Database off of the public internet.

ATP – Database Actions – Access

In the ATP page of the new created Autonomous Database, a click on Database Access shows you the URL to the web console. This URL cannot be resolved from your local workstation, the hostname is unknown. The hostname has to be resolved first by your DNS (quick and dirty: local hosts file) or you use the IP address instead which is provided in the Autonomous Database Details page.

The private endpoint IP 192.168.201.11 is in our OCI private subnet. I have added it temporarily to my local workstation hosts file.

ATP – Database Actions – User ADBSNMP

Login into Database Actions with the provided URL to enable the already existing user ADBSNMP for monitoring.

  • User: ADMIN
  • Password: Your ADB password

In the Database Actions Launchpad, select DATABASE USERS.

Edit user ADBSNMP, unlock the account and set a password. This password is later used in Oracle Enterprise Manager 13c.

ATP – Download Wallet

On the Autonomous Database Details page, download the DB Connection wallet. The wallet and the wallet password are later used in Oracle Enterprise Manager 13c.

Oracle Enterprise Manager 13c – ATP Host Name Resolution

Oracle uses FQDN in the ADB wallet tnsnames.ora file. Verify if the hostname which is listed above on the details page as Private Endpoint can be resolved by the on-prem OEM host. Alternative: Add this line to the /etc/hosts file. Attention: this is not the same domain as used for the Database Actions console (oraclecloudapps.com vs. oraclecloud.com). Tip for troubleshooting: Test the database connection with your local SQL Developer and user ADBSNMP, for a local SQL*Plus connect you have to extract the wallet file and configure Oracle Net.

Connectivity check firewall and routing with Telnet:

Oracle Enterprise Manager 13c – Add Targets Manually

Login as SYSMAN – SetupAdd TargetAdd Targets Manually.

Add Non-Host Targets Manually.

As Agent Host, you can choose your local agent which is running on the OEM host. Select target type for Autonomus Data Warehouse or Autonomus Transaction Processing.

  • Set Target Name, choose the OCI Client Credential Wallet and set Monitoring Password.
  • Test the connection.
  • Next and Submit.

The target will be added in the next minutes.

Oracle Enterprise Manager 13c – ADB Detail Page

The Autonomous Database is added as new database target and can be monitored on the same way as the on-prem databases. As it is a service, the metrics are limited. For example: when the ADB is stopped, the Oracle Enterprise Manager shows an Availability Evaluation Error.

Summary

Adding an Oracle Cloud Infrastructure Autonomous Database in an on-prem Oracle Enterprise Manager EM13c by using a private endpoint is a great thing. Take care about the routing and the firewall rules, verify all connections before you add an ADB as a new target. Even when the metrics are limited, for a basic monitoring (+) is it ok.

Next step: I want to deploy an  EM13c agent on a DBCS system. The MOS note DBCS: How to Deploy EM Agent on Cloud / DBCS Instance (Doc ID 2400965.1) shows the old OCI classic way, a SR to ask if this is allowed in OCI Gen. 2 too is already raised.

This was the last blog post of the year 2021 – see you in 2022! #happynewyear

Oracle Enterprise Manager 13c Release 5 – Time for Release Update 2

Since a few days the 13.5.0.2 Enterprise Manager Cloud Control Base Platform Monthly Release Update (RU) 2 is available to apply. Time to update my 13.5 lab environment at home to get a first experience about a lot of new features like Oracle Cloud Infrastructure Resource Discovery, Oracle Enterprise Manager Dashboards, Database Scheduler Jobs Metrics and many more.

Notes and Links

My Oracle Support Notes

  • 13.5.0 Enterprise Manager Cloud Control Base Platform Monthly Release Update (RU) 2 (Doc ID 2822316.1)
  • Enterprise Manager 13.5 Main Release Update List (Includes Plug-ins) (Doc ID 2760230.2)

Other Links

Patch Number 33456001

The Environment

  • Oracle Enterprise Manager 13.5 running on Oracle Linux 7.9
  • Oracle Enterprise Edition Repository 19.13 Single Tenant
  • Oracle Restart / ASM 21.0.0
  • Additional Oracle Linux Server with 19.3.0 Container Databases
  • All targets up and running
  • My Oracle Support connected
  • Patch software staged directory in /u01/app/oracle/stage/33456001

Installed Plugins

Verification by emcli. Plugins actually have release number 13.5.1.0.0.

 

Prerequisites

For the readme:

  1. On the Repository Database, apply Database Release Update 19.11.0.0.0 Patch 32545013 and Overlay patch for 17777718 on top of Database Release Update 19.11.0.0.0. Or apply Database Release Update 19.12.0.0.0 Patch 32904851 or its later Database Release Update version patch.

  2. On Windows, apply Oracle Repository Creation Utility patch 33053642 on all OMSes.

  3. Ensure that you have the latest version of OMSPatcher version 13.9.5.1.0 for Enterprise Manager 13.5.0.0 release on all OMSes.

In my case I had to update OMSPatcher and his little sister OPatch. For upgrading these components, I recommend these notes:

  • 13.5: How To Upgrade Enterprise Manager 13.5 Cloud Control OMSPatcher Utility to Version 13.9.5.1.0 (Doc ID 2809842.1)
  • Patch 28186730: OPATCH 13.9.4.2.7 FOR EM 13.4, 13.5 AND FMW/WLS 12.2.1.3.0, 12.2.1.4.0 AND 14.1.1.0.0

OMSPatcher

OPatch

Software Directory

The release update is transferred to the local stage directory:

Pre-Check

Run the pre-check to analyze the existing Oracle Enterprise Manager system.

These warning messages about patches which can not applied can be ignored, I don’t have installed these plugins like SMF (Enterprise Manager for Fusion Applications):

Apply

1st – Stop the Oracle Management Server

2nd – Apply RU with OMSPatcher

As in some blog posts before, I don’t use a credential file, therefore the username (if not using weblogic which is default) and the password for the Weblogic AdminServer has to be provided. The job runs some minutes. The OMS will be started automatically.

Verification

Get the Release Update Information, the inventory was updated:

Next Step

  • Plugin Upgrade
  • Agent Patch

Summary

Some weeks ago there was a presentation with all these new features like customizable dashboards. The update process himself was easy and straight forward as we knew from other releases. Finally the Release Update 2 is arrived. Let’s see in the next days what’s in the box.

Oracle Cloud Infrastructure Resource Naming Conventions – A short Friday Blog Post

Cleaning up the OCI Resource Chaos

This week I have removed all my Oracle Cloud Infrastructure lab resources and cleaned up my compartment to start from scratch building environments with the Automation Manager. There were a lot of resources with names like webserver01, vcn-lab-01, block-volume-web-clone-47 and so on. When I have realized this naming chaos, I have decided to spend some minutes today Friday to think about a small naming convention for my future Oracle Cloud Infrastructure projects.

Why having a naming convention? Wikipedia says about Naming Convention:

In computer programming, a naming convention is a set of rules for choosing the character sequence to be used for identifiers which denote variables, types, functions, and other entities in source code and documentation.

Sounds good, this is what I need. There are some proposal docs in the web about naming convention from Azure and TrendMicro – I used them as an inspiration.

Naming Convention

To identify an OCI resource, tagging is not enough. I like to see it at a glance what a resource is doing, which project and, if required, some additional information. This here is just a idea how it could work. Any other ideas are welcome :-). I have decided to define a resource by:

  • A resource type like bv for Block Volumes
  • A region key – based on https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm
  • An environment code – like letter t for test, d for development and p for production
  • A stack – this could be a project name or a purpose
  • A instance – the number of the resource if more than one is created

And in some cases where this information is not enough:

  • An additional –  like read-only, protected etc.

Examples

Resource Name Purpose
vcn-zrh-t-newcrm-001 VCN, region Zurich, test, NEWCRM project
sn-prv-fra-p-transfer-001 Private Subnet, region Frankfurt, production, TRANSFER project
nsg-zrh-d-ipsec-001 Network Security Group, region Zurich, development, IPSEC project
bv-zrh-t-ipsec-olvm-engine-001 Block Volume, region Zurich, test, IPSEC project, compute instance olvm-engine
log-ams-p-crm-read-001 Log Group ,region Amsterdam, production, CRM project, read only
ci-fra-t-science-ngnix-003 Compute Instance , region Marseille, test, SCIENCE project, NGINX server

 

A first View – Network Visualizer

This looks not so bad.

Summary

If you want to bring an order into your naming chaos, then you have to define a naming convention. There are for sure many other ideas around the world. How do you handle it? Just drop me a message by LinkedIn or Twitter and let me knolw if I can add your idea here in this blog too.

Oracle 21c runInstaller on Oracle Linux 8 – Don’t forget to run orainstRoot.sh – [FATAL] [INS-32035]

In one of my lab environments where an Oracle 19c database is up and running, it was time to upgrade it to 21c by AutoUpgrade. The new ORACLE_HOME was ready, the software extracted. But when running the OUI installer in silent mode, it stops working: [FATAL] [INS-32035].

runInstaller – 1st run

The error from the OUI logfile:

Quick Solution

An opatch lsinventory command from the existing 19c installation shows, that there is a file called oraInst.loc is located in the 19c ORACLE_HOME. But for existing RDBMS installations, the OUI has to verify the inventory location in /etc/oraInst.loc or depending on the OS in  /var/opt/oracle/oraInst.loc. But: NO FILE THERE.

The quick and simple solution was as OS user root to copy the existing file which points to the inventory directory to /etc and re-start the OUI.

runInstaller – 2nd run

Why this happened?

During the old 19c installation, I did not run /u01/app/oraInventory/orainstRoot.sh which creates this file. Extract from the 19c runInstaller log:

Alternative solution

Just run /u01/app/oraInventory/orainstRoot.sh again – it creates the file /etc/oraInst.loc for you and sets the correct permissions. Take a look here in this My Oracle Support note  for a detailed description what the script does:

  • What Are Root.sh And OrainstRoot.sh Scripts In A Standalone RDBMS Installation? (Doc ID 1493121.1)

Summary

This was really a stupid error. Lesson learned: read the f* terminal output – even in lab environments. AutoUpgrade can start now 🙂

Oracle Cloud Infrastructure – one DRG to many VCNs – arrived in Zurich / Switzerland – Upgrade Time!

One of my OCI favorites is finally arrived in Oracle Cloud Infrastructure data center eu-zurich-1 – the possibility to attach one Dynamic Routing Gateway DRG to multiple Virtual Cloud Networks VCNs. To see how it works, Simo has written three blog posts where it’s very well described. Now it’s time to upgrade my existing DRG!

Oracle’s OCI documentation:

How to upgrade an existing DRG? See here – just click on the images to see the details.

Step 1 – verify on an existing DRG if your region is ready – if yes, press the Upgrade DRG button

As you can see here, in this version of DRG, there is the existing IPSec connection listed.

Step 2 – Do you really want to upgrade the DRG? Sure!

Step 3 – The UI View changes, new Resources are created

After some minutes, the DRG view has changed, now the Virtual Cloud Network attachments are visible. New resources like DRG Route Tables and Route Distributions are created.

Step 4 – What about the existing VPN Tunnel?

Verify if the existing tunnel is still running and try to connect to be sure that the routing work properly. It works – nice!

Summary

To attach a single DRG to multiple VCNS opens new possibilities in network design. Read the manual properly to understand how components like DRG Route Tables and DRG Attachments are working. Well done, well done, Oracle!