martinberger.com – Deheime@Jurasüdfuss.SO.CH

Latest Posts

OCI Cloud Performance Management for On-Premises Databases – Part 2 – Database Configuration

Author: Martin Berger 8. March 2021

In the part 1 of this blog post series, we have installed the Management Agent. Now it’s time to add the database. For this case an agent plug-in has to be installed first before we can configure the database.

More information about the management agent administration: https://docs.oracle.com/en-us/iaas/management-agents/doc/management-agents-administration-tasks.html

This is a small blog post series

Part 1 shows the installation of the Management Agent on an on-premises system
Part 2 describes how to manage this Oracle database in OCI
Part 3 watches behind the scenes for billing, job execution etc. – coming soon…

My Setup

An OCI Tenant in datacenter EU-FRANKFURT-1
An empty compartment called datacenter-kestenholz
An on-premises database called CDB114, running on Oracle Linux 7
OCI Management Agent up and running

The goal is to handle the on-premises database in OCI. Output from the Trivadis TVD-Basenv(TM) framework which show the database up and running:

TYPE (Cluster|DG)  : SID/PROCESS  STATUS      HOME      [2021-02-22 08:05:03]
-----------------------------------------------------------------------------
Dummy rdbms_ee     : rdbms19      n/a         /u01/app/oracle/product/19.0.0/dbhome_1

DB-instance (N|N)  : CDB144       open        /u01/app/oracle/product/19.0.0/dbhome_1

Listener           : LISTENER     up          /u01/app/oracle/product/19.0.0/dbhome_1

TYPE (Cluster|DG) : SID/PROCESS STATUS HOME [2021-02-22 08:05:03]

-----------------------------------------------------------------------------

Dummy rdbms_ee : rdbms19 n/a /u01/app/oracle/product/19.0.0/dbhome_1

DB-instance (N|N) : CDB144 open /u01/app/oracle/product/19.0.0/dbhome_1

Listener : LISTENER up /u01/app/oracle/product/19.0.0/dbhome_1

Service Plug-In Deployment

From the agent page, we select the Management Agent to see the details. Click on Deploy Plug-Ins.

Select the plug-in for the database management, press Update to deploy it on the management agent.

Now you see the confirmation that the deployment process is initiated.

After some seconds, you can see on the top right of the agent overview page, that the service plug-in is installed.

Register the External Database

In External Database menu, we register an external container database first. Be sure that the right compartment is selected.

Set the compartment and display name and click Register.

More about this process: https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/externalcreating.htm

Connect to the External Database

After registration, the container database is added, but not connected. In this step we create a connector to setup the database in OCI.

Set display name, DNS hostname, port and service first.

The connector type cannot be changed, select the management agent from the connector agent id list, specify database connection credentials. I use the database schema SYSTEM. Click on Connect to External Container Database.

After some minutes, the work request is done and the database is connected. The database is in state Available.

We repeat the steps above to add the three external Pluggable Databases on the same way.

Enable Database Management

Before Database Management from OCI can be used, it has to be enabled. This is done on the page where the external Container Database is visible. According Oracle’s license handling, this has to be done manually. You can select between BYOL and License Included. After some minutes, the database management is enabled and the agent is updated. More about licensing and pricing in the next blog post number 3 which is coming soon.

Database Management – Database Groups and Fleet Summary

When Database Management is enabled in OCI for all Container and Pluggable Databases, the management agent starts to collect database related data. We can organized the external databases in groups.

In Database Management menu, create a new database group. In my case I named it Datacenter_Kestenholz which contains the previous added external databases.

When Database Management is enabled in OCI for all Container and Pluggable Databases, the Management Agent starts to collect database related data. Some moments later, the dashboard has the first values.

Note: The Container Database is not shown in the Fleet Summary dashboard at the moment, only Pluggable Databases are listed with performance data etc. – I will open a SR to clarify it.

Summary Part 2

When the Management Agent is up and running, adding a database is straight forward. But we have to clarify the license situation first. More about licensing and cost in the next blog post part 3.

Uncategorized

Oracle Cloud Infrastructure – A small and secure Development Environment – Next Level: Terraform

Author: Martin Berger 1. March 2021

In a previous blog post I wrote how to build a small and secure development environment in Oracle Cloud Infrastructure with an OpenVPN entry point and a compute instance in a private setup. Now there is the Terraform code available in GitHub to setup it on an easy and reusable way:

terraform-examples/oci/openvpnas at main · Trivadis/terraform-examples (github.com)

What you get

After executing the code, you will get this setup here:

an OpenVPN Access Server from OCI Marketplace
a Compute Instance

Prerequisites

Oracle OCI CLI installed and configured
Terraform up and running
Git client installed

SSH Key Access

An example private and public SSH key to get access on the compute instance in the private subnet is provided in subdirectory SSH, if you want to use your own SSH key – which is highly recommended – just replace the public key variable in file variables.tf with your own key:

variable "ssh_public_key" {
	default = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCbytm9y7UigHeV2L0vUXWqFiplf9ntG9VMUBGwEoWATV6Ir/4udvObm/6dFCltVmvHVWD5XdbXWvyz9is69jH3Cb2hOUtyZMeXJBTtXnMuC0HaN7zHmrV6qfkQDiRpJNHCEpD3LhAL2VG7tViqCC9rSTEfezKibjGXVl1R606xp57oduuT1V4g82+BLYKdEsDAfgVLI8z23dSYyzd3Kb6ikqG+9wSA1KWWb051KE8ofRtL+FD5cZ/uGLwhczIbMaEZjHs5Zv5L9kWKUU4nBIxv4RN2QjbpFQ+EoTVVZqPeT1eILKEuOFPy5s42AA1an4FMdSoLmEuRtC0sIoR5L5kj imported-openssh-key"
  type        = string
}

variable "ssh_public_key" {

default = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCbytm9y7UigHeV2L0vUXWqFiplf9ntG9VMUBGwEoWATV6Ir/4udvObm/6dFCltVmvHVWD5XdbXWvyz9is69jH3Cb2hOUtyZMeXJBTtXnMuC0HaN7zHmrV6qfkQDiRpJNHCEpD3LhAL2VG7tViqCC9rSTEfezKibjGXVl1R606xp57oduuT1V4g82+BLYKdEsDAfgVLI8z23dSYyzd3Kb6ikqG+9wSA1KWWb051KE8ofRtL+FD5cZ/uGLwhczIbMaEZjHs5Zv5L9kWKUU4nBIxv4RN2QjbpFQ+EoTVVZqPeT1eILKEuOFPy5s42AA1an4FMdSoLmEuRtC0sIoR5L5kj imported-openssh-key"

type = string

}

Some Code Snippets

Terraform State File

In file backend.tf, the Terraform state is set to local, there is also an example to store your state file in OCI Object Store. Please prepare the bucket first according the documentation here: Using Object Storage for State Files (oracle.com). Example:

# define remote state file for terraform
terraform {
  required_version = ">= 0.13.0"
  backend "http" {
    update_method = "PUT"
    address       =  "https://objectstorage.eu-zurich-1.oraclecloud.com/p/............./b/terraform_state_file/o/terraform.tfstate"
  }
}

# define remote state file for terraform

terraform {

required_version = ">= 0.13.0"

backend "http" {

update_method = "PUT"

address = "https://objectstorage.eu-zurich-1.oraclecloud.com/p/............./b/terraform_state_file/o/terraform.tfstate"

}

Compute Instance Image

The compute instance as defined in compute.tf uses this images according your location – for other data centers or images, follow here is the link where all images are listed: https://docs.us-phoenix-1.oraclecloud.com/images/

variable "linux_image_ocid" {
  type = map(any)

  default = {
    # See https://docs.us-phoenix-1.oraclecloud.com/images/
    # Oracle-provided image "Oracle-Linux-7.8-2020.04.17-0"
    eu-zurich-1    = "ocid1.image.oc1.eu-zurich-1.aaaaaaaa5ganyj57k2dqyik4m4btpuq23le3e7clh56rjhgz6fekvtoyazqa"
    eu-frankfurt-1 = "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaavz6p7tyrczcwd5uvq6x2wqkbwcrjjbuohbjomtzv32k5bq24rsha"
    eu-amsterdam-1 = "ocid1.image.oc1.eu-amsterdam-1.aaaaaaaaie5km236l53ymcvpwufyb2srtc3hw2pa6astfjdafzlxxdv5nfsq"
    us-ashburn-1   = "ocid1.image.oc1.iad.aaaaaaaahjkmmew2pjrcpylaf6zdddtom6xjnazwptervti35keqd4fdylca"
  }
}

variable "linux_image_ocid" {

type = map(any)

default = {

# See https://docs.us-phoenix-1.oraclecloud.com/images/

# Oracle-provided image "Oracle-Linux-7.8-2020.04.17-0"

eu-zurich-1 = "ocid1.image.oc1.eu-zurich-1.aaaaaaaa5ganyj57k2dqyik4m4btpuq23le3e7clh56rjhgz6fekvtoyazqa"

eu-frankfurt-1 = "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaavz6p7tyrczcwd5uvq6x2wqkbwcrjjbuohbjomtzv32k5bq24rsha"

eu-amsterdam-1 = "ocid1.image.oc1.eu-amsterdam-1.aaaaaaaaie5km236l53ymcvpwufyb2srtc3hw2pa6astfjdafzlxxdv5nfsq"

us-ashburn-1 = "ocid1.image.oc1.iad.aaaaaaaahjkmmew2pjrcpylaf6zdddtom6xjnazwptervti35keqd4fdylca"

}

OpenVPN Marketplace Image

variable "mp_listing_id" {
  description = "OCI Marketplace Listing ID"
  default     = "ocid1.appcataloglisting.oc1..aaaaaaaafbgwdxg5j6jnyfhbcxvd62iabcraaf6bwu2u2nhrddztrrle66lq"
  type        = string
}

variable "mp_listing_resource_id" {
  description = "OCI Marketplace Listing Resource ID"
  default     = "ocid1.image.oc1..aaaaaaaa4ozqggnywlp3e3wzvu5x3aoohkt6cwm2pumgpn2tlzroj756azma"
  type        = string
}

variable "mp_listing_resource_version" {
  description = "OCI Marketplace Listing Version"
  default     = "AS_2.8.3"
  type        = string
}

variable "mp_listing_id" {

description = "OCI Marketplace Listing ID"

default = "ocid1.appcataloglisting.oc1..aaaaaaaafbgwdxg5j6jnyfhbcxvd62iabcraaf6bwu2u2nhrddztrrle66lq"

type = string

}

variable "mp_listing_resource_id" {

description = "OCI Marketplace Listing Resource ID"

default = "ocid1.image.oc1..aaaaaaaa4ozqggnywlp3e3wzvu5x3aoohkt6cwm2pumgpn2tlzroj756azma"

type = string

}

variable "mp_listing_resource_version" {

description = "OCI Marketplace Listing Version"

default = "AS_2.8.3"

type = string

}

Let’s Terraform it

0: Clone GitHub Directory

And go to openvpnas subdirectory.

$ git clone https://github.com/Trivadis/terraform-examples.git
Cloning into 'terraform-examples'...
remote: Enumerating objects: 241, done.
remote: Counting objects: 100% (241/241), done.
remote: Compressing objects: 100% (155/155), done.
remote: Total 241 (delta 130), reused 155 (delta 72), pack-reused 0
Receiving objects: 100% (241/241), 230.16 KiB | 0 bytes/s, done.
Resolving deltas: 100% (130/130), done.
Checking connectivity... done.

$ cd terraform-examples/oci/openvpnas

$ git clone https://github.com/Trivadis/terraform-examples.git

Cloning into 'terraform-examples'...

remote: Enumerating objects: 241, done.

remote: Counting objects: 100% (241/241), done.

remote: Compressing objects: 100% (155/155), done.

remote: Total 241 (delta 130), reused 155 (delta 72), pack-reused 0

Receiving objects: 100% (241/241), 230.16 KiB | 0 bytes/s, done.

Resolving deltas: 100% (130/130), done.

Checking connectivity... done.

$ cd terraform-examples/oci/openvpnas

1st: Set Variables

export TF_VAR_tenancy_ocid=<your_tenancy_ocid>
export TF_VAR_user_ocid=<your_username_OCID>                              
export TF_VAR_private_key_path=<your_ssh_private_key>   
export TF_VAR_fingerprint=<your_public_key_fingerprint>
export TF_VAR_region=<your_OCI_region>                           
export TF_VAR_compartment_name=<your_compartment_name>
export TF_VAR_compartment_description=<your_compartment_description>
export TF_VAR_compartment_master_ocid=<your_OCID of the master compartment>
export TF_VAR_openvpn_admin_password=<your_openvpn_inital_password_for_user_openvpnadmin>

export TF_VAR_tenancy_ocid=<your_tenancy_ocid>

export TF_VAR_user_ocid=<your_username_OCID>

export TF_VAR_private_key_path=<your_ssh_private_key>

export TF_VAR_fingerprint=<your_public_key_fingerprint>

export TF_VAR_region=<your_OCI_region>

export TF_VAR_compartment_name=<your_compartment_name>

export TF_VAR_compartment_description=<your_compartment_description>

export TF_VAR_compartment_master_ocid=<your_OCID of the master compartment>

export TF_VAR_openvpn_admin_password=<your_openvpn_inital_password_for_user_openvpnadmin>

2nd: terraform init, plan and apply

$ terraform init
$ terraform plan
$ terraform apply

$ terraform init

$ terraform plan

$ terraform apply

Login and Go!

And after some minutes – you can get access to the OpenVPN Administrator Dashboard or get your client or profile. All required information like OpenVPN Access Server public IP, URL etc. are provided in the Terraform output.

Login into the compute instance with the private key and the private subnet IP address when the VPN tunnel is up and running:

Links and Documents

A good point to start: Terraform: Set Up a Simple Infrastructure with OCI Terraform (oracle.com)
Getting started with the Terraform OCI Provider: Getting Started (oracle.com)
Hashicorp Documentation: Docs overview | hashicorp/oci | Terraform Registry
Blog post by Lucas Jellema with the first steps: Very first steps in Oracle Cloud Infrastructure as Code with Terraform – AMIS, Data Driven Blog – Oracle & Microsoft Azure

Summary

Setup an Oracle Cloud Infrastructure with Terraform is a good way to start in the IaC – Infrastructure as Code – world. Feel free to use this code a base for your next project. What’s your next level? Mine is to integrate the code in the Oracle Cloud Resource Manager – stay tuned!

Uncategorized

OCI Cloud Performance Management for On-Premises Databases – Part 1 – Management Agent Installation

Author: Martin Berger 22. February 2021

The OCI Management Agent service collects data from services and sources for monitoring and management in Oracle Cloud Infrastructure. In this blog post series I will show you how you can monitor and manage an on-premises Oracle databases in OCI. The communication between an agent and OCI requires an Agent Install Key and is based on HTTPS. Service Plugins extend a Management Agent for example for Oracle database performance monitoring and management or log analytics.

This is a small blog post series

Part 1 shows the installation of the Management Agent on an on-premises system
Part 2 describes how to manage this Oracle database in OCI
Part 3 watches behind the scenes for billing, job execution etc.

My Setup

An OCI Tenant in datacenter EU-FRANKFURT-1
An OCI compartment called datacenter-kestenholz
An on-premises Container Database called CDB114, running on Oracle Linux 7
Three on-premises Pluggable Databases

The goal is to manage the on-premises database in Oracle Cloud Infrastructure OCI. Output from the Trivadis TVD-Basenv(TM) framework which show the database up and running:

TYPE (Cluster|DG)  : SID/PROCESS  STATUS      HOME      [2021-02-22 08:05:03]
-----------------------------------------------------------------------------
Dummy rdbms_ee     : rdbms19      n/a         /u01/app/oracle/product/19.0.0/dbhome_1

DB-instance (N|N)  : CDB144       open        /u01/app/oracle/product/19.0.0/dbhome_1

Listener           : LISTENER     up          /u01/app/oracle/product/19.0.0/dbhome_1

TYPE (Cluster|DG) : SID/PROCESS STATUS HOME [2021-02-22 08:05:03]

-----------------------------------------------------------------------------

Dummy rdbms_ee : rdbms19 n/a /u01/app/oracle/product/19.0.0/dbhome_1

DB-instance (N|N) : CDB144 open /u01/app/oracle/product/19.0.0/dbhome_1

Listener : LISTENER up /u01/app/oracle/product/19.0.0/dbhome_1

Prerequisites for Management Agent Installation

Oracle Linux 6 or higher
Red Hat Enterprise Linux 6 or higher
CentOS 6 or 7
SUSE Linux Enterprise Server 12 or 15
Windows Server 2012 R2, 2016 or 2019

There are other prerequisites on the target server like the correct Java version (e.g. version 11 does not work) and sudo permissions. For the complete list of prerequisites, see here:

https://docs.oracle.com/en-us/iaas/management-agents/doc/perform-prerequisites-deploying-management-agents.html#GUID-BC5862F0-3E68-4096-B18E-C4462BC76271

Setup for OCI Management Agent

It is recommended to handle the agents in a separate user group and with policies. This allows us to define the Management Agent management on an fine granular level.

Group AGENT_ADMINS

According the documentation, I have created an user group called AGENT_ADMINS.

Policy Datacenter_Kestenholz_Agent_Policy

A new policy is created that allows the admin group to interact with the management agents, handle keys etc.

ALLOW GROUP AGENT_ADMINS TO MANAGE management-agents IN COMPARTMENT datacenter-kestenholz
ALLOW GROUP AGENT_ADMINS TO MANAGE management-agent-install-keys IN COMPARTMENT datacenter-kestenholz
ALLOW GROUP AGENT_ADMINS TO READ METRICS IN COMPARTMENT datacenter-kestenholz
ALLOW GROUP AGENT_ADMINS TO READ USERS IN TENANCY

ALLOW GROUP AGENT_ADMINS TO MANAGE management-agents IN COMPARTMENT datacenter-kestenholz

ALLOW GROUP AGENT_ADMINS TO MANAGE management-agent-install-keys IN COMPARTMENT datacenter-kestenholz

ALLOW GROUP AGENT_ADMINS TO READ METRICS IN COMPARTMENT datacenter-kestenholz

ALLOW GROUP AGENT_ADMINS TO READ USERS IN TENANCY

Dynamic Group Management_Agent_Dynamic_Group

New added agents in the compartment belong automatically to this group. Replace the OCID with the OCID for your compartment.

Policy Datacenter_Kestenholz_Agent_Communication_Policy

A policy is required that allows the agents to communicate with the OCI endpoints. This policy is important, otherwise you run in an communication error (see below in section troubleshooting).

ALLOW DYNAMIC-GROUP Management_Agent_Dynamic_Group TO MANAGE management-agents IN COMPARTMENT datacenter-kestenholz
ALLOW DYNAMIC-GROUP Management_Agent_Dynamic_Group TO USE METRICS IN COMPARTMENT datacenter-kestenholz

1 2	ALLOW DYNAMIC-GROUP Management_Agent_Dynamic_Group TO MANAGE management-agents IN COMPARTMENT datacenter-kestenholz ALLOW DYNAMIC-GROUP Management_Agent_Dynamic_Group TO USE METRICS IN COMPARTMENT datacenter-kestenholz

Install On-Premises Management Agent

Create Agent Install Key

Go to Management Agent Menu / Downloads and Keys, create a new Agent Install Key. Set the compartment and the time how long the key is valid. In this example, I need to replace the key after one month.

When click on Download Key to File, a textfile is created with the ManagementAgentInstallKey and all other (optional) parameters which can be used for install. You can use this file as responsefile template later.

Download the Software and Transfer it to the Target Server

I use the Agent for Linux and transferred it to the target on-premises server into a stage directory as OS user root.

[root@bifangstrasse stage]# ll
total 8128
-rw-r--r--. 1 root root 8323072 Feb 21 20:38 oracle.mgmt_agent.rpm

[root@bifangstrasse stage]# ll

total 8128

-rw-r--r--. 1 root root 8323072 Feb 21 20:38 oracle.mgmt_agent.rpm

Create a Local Response File

This is an example of a simple two-lines response file for agent installation in the same folder where the rpm is located, called input.rsp. The parameter managementAgentInstallKey is visible in the OCI web interface, the CredentialWalletPassword is your password for the wallet.

managementAgentInstallKey = MS4wLHVzLWFzaGJ1cm4tMSxvY2lkMS50ZW5hbmN5-<YOUR_AGENT_KEY_HERE>
CredentialWalletPassword = W3lcome#1

1 2	managementAgentInstallKey = MS4wLHVzLWFzaGJ1cm4tMSxvY2lkMS50ZW5hbmN5-<YOUR_AGENT_KEY_HERE> CredentialWalletPassword = W3lcome#1

RPM Installation

As OS user root (or a user with sudo permissions) – install the rpm file. Here you can see that the minimum required Java version is not met.

[root@bifangstrasse stage]# 
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Checking pre-requisites
        Checking if any previous agent service exists
        Checking if OS has systemd or initd
        Checking available disk space for agent install
        Checking if /opt/oracle/mgmt_agent directory exists
        Checking if 'mgmt_agent' user exists
        Checking agent version
        Checking Java version
                JAVA_HOME is not set or not readable to root
                Trying default path /usr/bin/java
                Agent only supports JDK 8 with a miniumum upgrade version JDK 8u162. Please set your preferred path in JAVA_HOME
error: %prein(oracle.mgmt_agent-210114.0548-1.x86_64) scriptlet failed, exit status 1
error: oracle.mgmt_agent-210114.0548-1.x86_64: install failed

[root@bifangstrasse stage]#

Verifying... ################################# [100%]

Preparing... ################################# [100%]

Checking pre-requisites

Checking if any previous agent service exists

Checking if OS has systemd or initd

Checking available disk space for agent install

Checking if /opt/oracle/mgmt_agent directory exists

Checking if 'mgmt_agent' user exists

Checking agent version

Checking Java version

JAVA_HOME is not set or not readable to root

Trying default path /usr/bin/java

Agent only supports JDK 8 with a miniumum upgrade version JDK 8u162. Please set your preferred path in JAVA_HOME

error: %prein(oracle.mgmt_agent-210114.0548-1.x86_64) scriptlet failed, exit status 1

error: oracle.mgmt_agent-210114.0548-1.x86_64: install failed

After installing the jdk-8u281-linux-x64.rpm to update the server Java version, the installer runs fine.

[root@bifangstrasse stage]# rpm -Uhv jdk-8u281-linux-x64.rpm
warning: jdk-8u281-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:jdk1.8-2000:1.8.0_281-fcs        ################################# [100%]
Unpacking JAR files...
        tools.jar...
        plugin.jar...
        javaws.jar...
        deploy.jar...
        rt.jar...
        jsse.jar...
        charsets.jar...
        localedata.jar...

[root@bifangstrasse stage]# java -version
java version "1.8.0_281"
Java(TM) SE Runtime Environment (build 1.8.0_281-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.281-b09, mixed mode)

[root@bifangstrasse stage]# rpm -Uhv jdk-8u281-linux-x64.rpm

warning: jdk-8u281-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY

Verifying... ################################# [100%]

Preparing... ################################# [100%]

Updating / installing...

1:jdk1.8-2000:1.8.0_281-fcs ################################# [100%]

Unpacking JAR files...

tools.jar...

plugin.jar...

javaws.jar...

deploy.jar...

rt.jar...

jsse.jar...

charsets.jar...

localedata.jar...

[root@bifangstrasse stage]# java -version

java version "1.8.0_281"

Java(TM) SE Runtime Environment (build 1.8.0_281-b09)

Java HotSpot(TM) 64-Bit Server VM (build 25.281-b09, mixed mode)

2nd try – successful

[root@bifangstrasse stage]# rpm -ihv oracle.mgmt_agent.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Checking pre-requisites
        Checking if any previous agent service exists
        Checking if OS has systemd or initd
        Checking available disk space for agent install
        Checking if /opt/oracle/mgmt_agent directory exists
        Checking if 'mgmt_agent' user exists
        Checking agent version
        Checking Java version
                JAVA_HOME is not set or not readable to root
                Trying default path /usr/bin/java
                Java version: 1.8.0_281 found at /usr/bin/java
Updating / installing...
   1:oracle.mgmt_agent-210114.0548-1  ################################# [100%]

Executing install
        Unpacking software zip
        Copying files to destination dir (/opt/oracle/mgmt_agent)
        Initializing software from template
        Creating 'mgmt_agent' daemon
        Agent Install Logs: /opt/oracle/mgmt_agent/installer-logs/installer.log.0

        Setup agent using input response file (run as any user with 'sudo' privileges)
        Usage:
                sudo /opt/oracle/mgmt_agent/agent_inst/bin/setup.sh opts=[FULL_PATH_TO_INPUT.RSP]

Agent install successful

[root@bifangstrasse stage]# rpm -ihv oracle.mgmt_agent.rpm

Verifying... ################################# [100%]

Preparing... ################################# [100%]

Checking pre-requisites

Checking if any previous agent service exists

Checking if OS has systemd or initd

Checking available disk space for agent install

Checking if /opt/oracle/mgmt_agent directory exists

Checking if 'mgmt_agent' user exists

Checking agent version

Checking Java version

JAVA_HOME is not set or not readable to root

Trying default path /usr/bin/java

Java version: 1.8.0_281 found at /usr/bin/java

Updating / installing...

1:oracle.mgmt_agent-210114.0548-1 ################################# [100%]

Executing install

Unpacking software zip

Copying files to destination dir (/opt/oracle/mgmt_agent)

Initializing software from template

Creating 'mgmt_agent' daemon

Agent Install Logs: /opt/oracle/mgmt_agent/installer-logs/installer.log.0

Setup agent using input response file (run as any user with 'sudo' privileges)

Usage:

sudo /opt/oracle/mgmt_agent/agent_inst/bin/setup.sh opts=[FULL_PATH_TO_INPUT.RSP]

Agent install successful

Agent Configuration

Run the install script with the created response file as additional parameter. The agent will be started automatically.

[root@bifangstrasse stage]# /opt/oracle/mgmt_agent/agent_inst/bin/setup.sh opts=/stage/input.rsp

Executing configure

        Parsing input response file
        Generating communication wallet
        Validating install key
        Generating security artifacts
        Registering Management Agent

Starting agent...
Agent started successfully


Agent setup completed and the agent is running.
In the future agent can be started by directly running: sudo systemctl start mgmt_agent

Please make sure that you delete /stage/input.rsp or store it in secure location.

[root@bifangstrasse stage]# /opt/oracle/mgmt_agent/agent_inst/bin/setup.sh opts=/stage/input.rsp

Executing configure

Parsing input response file

Generating communication wallet

Validating install key

Generating security artifacts

Registering Management Agent

Starting agent...

Agent started successfully

Agent setup completed and the agent is running.

In the future agent can be started by directly running: sudo systemctl start mgmt_agent

Please make sure that you delete /stage/input.rsp or store it in secure location.

A systemd service is created.

[root@bifangstrasse stage]# systemctl list-units --type=service --state=active | grep mgmt_agent
mgmt_agent.service                 loaded active running mgmt_agent

1 2	[root@bifangstrasse stage]# systemctl list-units --type=service --state=active \| grep mgmt_agent mgmt_agent.service loaded active running mgmt_agent

Verify the Management Agent in Oracle Cloud Infrastructure

Immediately after the setup, the Management Agent is visible with status Active in OCI and starts uploading data.

Agent details:

Troubleshooting

Management Agent logfiles are located in directory /opt/oracle/mgmt_agent/agent_inst/log.

[root@bifangstrasse log]# ls -latr
total 64
drwxr-x---. 7 mgmt_agent mgmt_agent    68 Feb 21 21:23 ..
-rw-r-----. 1 mgmt_agent mgmt_agent     5 Feb 21 21:24 mgmt_agent.pid
-rw-r-----. 1 mgmt_agent mgmt_agent  3631 Feb 21 21:24 wrapper.log
-rw-r-----. 1 mgmt_agent mgmt_agent     0 Feb 21 21:24 mgmt_agent_errors.log
-rw-r-----. 1 mgmt_agent mgmt_agent     0 Feb 21 21:24 mgmt_agent_work.log
-rw-r-----. 1 mgmt_agent mgmt_agent    75 Feb 21 21:24 agent.pid
-rw-r-----. 1 mgmt_agent mgmt_agent  1679 Feb 21 21:24 startup.info
drwxr-x---. 2 mgmt_agent mgmt_agent   246 Feb 21 21:24 .
-rw-r-----. 1 mgmt_agent mgmt_agent     8 Feb 21 21:24 mgmt_agent.status
-rw-r-----. 1 mgmt_agent mgmt_agent     8 Feb 21 21:24 mgmt_agent.java.status
-rw-r-----. 1 mgmt_agent mgmt_agent 31966 Feb 21 21:25 mgmt_agent.log
-rw-r-----. 1 mgmt_agent mgmt_agent  5091 Feb 21 21:25 mgmt_agent_client.log

[root@bifangstrasse log]# ls -latr

total 64

drwxr-x---. 7 mgmt_agent mgmt_agent 68 Feb 21 21:23 ..

-rw-r-----. 1 mgmt_agent mgmt_agent 5 Feb 21 21:24 mgmt_agent.pid

-rw-r-----. 1 mgmt_agent mgmt_agent 3631 Feb 21 21:24 wrapper.log

-rw-r-----. 1 mgmt_agent mgmt_agent 0 Feb 21 21:24 mgmt_agent_errors.log

-rw-r-----. 1 mgmt_agent mgmt_agent 0 Feb 21 21:24 mgmt_agent_work.log

-rw-r-----. 1 mgmt_agent mgmt_agent 75 Feb 21 21:24 agent.pid

-rw-r-----. 1 mgmt_agent mgmt_agent 1679 Feb 21 21:24 startup.info

drwxr-x---. 2 mgmt_agent mgmt_agent 246 Feb 21 21:24 .

-rw-r-----. 1 mgmt_agent mgmt_agent 8 Feb 21 21:24 mgmt_agent.status

-rw-r-----. 1 mgmt_agent mgmt_agent 8 Feb 21 21:24 mgmt_agent.java.status

-rw-r-----. 1 mgmt_agent mgmt_agent 31966 Feb 21 21:25 mgmt_agent.log

-rw-r-----. 1 mgmt_agent mgmt_agent 5091 Feb 21 21:25 mgmt_agent_client.log

Example error when the policy for agent communication is not set properly:

Error:
<--> Endpoint:       management-agent.eu-frankfurt-1.oci.oraclecloud.com:-1
     opc-request-id: US3E76CLWXAJYY5YH6Y37BOULWGHCTT1
     StartTime:      2021-02-21 20:12:44,529 GMT
     Status:         404 Not Found
     Headers:        Content-Length=111
                     opc-request-id=US3E76CLWXAJYY5YH6Y37BOULWGHCTT1/01E0A8DF939076673DF971035E99E335/5BB6AA66C5811DC0CA68288F16327F47
                     Date=Sun, 21 Feb 2021 20:12:44 GMT
                     Content-Type=application/json
     ErrorBody:
{
  "code" : "NotAuthorizedOrNotFound",
  "message" : "Authorization failed or requested resource not found."
}
     EndTime:        2021-02-21 20:12:44,795 GMT

Error:

<--> Endpoint: management-agent.eu-frankfurt-1.oci.oraclecloud.com:-1

opc-request-id: US3E76CLWXAJYY5YH6Y37BOULWGHCTT1

StartTime: 2021-02-21 20:12:44,529 GMT

Status: 404 Not Found

Headers: Content-Length=111

opc-request-id=US3E76CLWXAJYY5YH6Y37BOULWGHCTT1/01E0A8DF939076673DF971035E99E335/5BB6AA66C5811DC0CA68288F16327F47

Date=Sun, 21 Feb 2021 20:12:44 GMT

Content-Type=application/json

ErrorBody:

{

"code" : "NotAuthorizedOrNotFound",

"message" : "Authorization failed or requested resource not found."

}

EndTime: 2021-02-21 20:12:44,795 GMT

From My Oracle Support: OCI : Management Agent Status Reporting As “Not Available” Post Installation (Doc ID 2745566.1)

Summary Part 1

The Management Agent installation and integration is easy to setup when all prerequisites are met. For troubleshooting you have full access on the agent logs. See you for blog post 2, where I try to integrate the on-premises Oracle databases into OCI.

Uncategorized

Oracle Cloud Infrastructure – A short Blog Post about a secure and small Development Setup

Author: Martin Berger 14. January 2021

For an internal project I had the pleasure to setup a new Oracle Cloud Infrastructure environment for an APEX development team. Here is a short overview about the setup.

Requirements

VPN Access from everywhere – 2 people are working maximal at same time on the environment
Oracle Standard Edition 2 – no license available in project
Small monitoring to verify server stats
Instances can be started and stopped from the developers to save costs for example over night, weekend, holiday etc.

Architecture Diagram

Resource	Network	Usage	Remarks
Open VPN Access Server	Public Subnet	VPN client access and traffic routing	OCI Cloud Marketplace Image – OpenVPN Access Server (2 FREE VPN Connections) – OpenVPN Inc. – Oracle Cloud Marketplace
Management Server	Private Subnet	OCI-CLI, Monitoring	Application server and database node start/stop with OCI-CLI, Grafana and Prometheus for monitoring
Application Server	Private Subnet	Tomcat ORDS, APEX	–
Database System	Private Subnet	–	OCI Database Standard Edition 2, Backup to Object Store enabled

Network Components

Regional private and public subnet
Security lists and network security groups
Private and public routing table
NAT gateway for regional private subnet

Monitoring

Grafana and Prometheus, running on the management server. The free shape VM.Standard.E2.1.Micro fits perfect for this small setup! The Prometheus node exporter runs on the database and the application server. I used this Grafana dashboard here: Prometheus Node Exporter Full dashboard for Grafana | Grafana Labs

Next Steps

Adding Influx DB for persistence
Adding the Oracle database to Grafana monitoring
Optimizing shape size for the database server according usage

Other Ideas

Create a blueprint for internal developer environments
Automate the setup with Terraform and Ansible

Summary

Setting up this infrastructure in Oracle Cloud Infrastructure was fun. All developer requirements are fulfilled. Started with the Network and OpenVPN configuration – I really like their Marketplace instance – and the moved on to application and database server, step-by-step. There are many other ideas what we can do more based on this setup, the work will not run out. #ilike

Oracle Cloud Infrastructure, Oracle Tools, Security, Testing, TrivadisContent

Oracle Cloud Infrastructure Data Safe – How to burn down 201.44 Swiss Francs in 30 Seconds…

Author: Martin Berger 11. January 2021

Is Data Safe really for free?

In the last autumn, the new Oracle Cloud Infrastructure feature called Data Safe was released. For sure, new features has to be tested. I have tested the Data Safe feature too and added a cloud database to Data Safe. But in my enthusiasm about this cool feature – or maybe it was just too late in the evening – I did a mistake by adding the database target. Four days later, I recognized that Data Safe is charged in my account. Mmm, but should it not be for free? First reaction: I raised an SR and described the case. The nice guy from My Oracle Support realized the situation quickly:

Dear Mister Berger, you have used the wrong target type when adding the Oracle Cloud Infrastructure database as a new Data Safe target.

From the Service Request:

B91632 – Oracle Cloud Infrastructure – Data Safe for Database Cloud Service – Each (Includes 1 million audit records per target per month) – Free
B91631 – Oracle Cloud Infrastructure – Data Safe for Database Cloud Service – Audit Record Collection Over 1 Million Records (over 1 million audit records per target per month) – 0.0800 / 10,000 Audit Records Per Target Per Month
B92733 – Oracle Cloud Infrastructure – Data Safe for On-Premises Databases – Target Database Per Month – 200.00 Target Database Per Month + Includes 1 million audit records per target per month (pre-requisite under B91632)

Indeed, indeed. According My Oracle Support I have used the wrong target type. Instead Oracle Cloud Database, I used Oracle Database on Compute. And did not realized, the mistake and ignored the text below to the dropdown box. Shame on me 😉 – here is the small, but important difference:

So far so good, the mistake was recognized. I deleted the target and added it from scratch with the correct target type. But this didn’t help, the charging went on.

Oracle Cloud Infrastructure Price List

Adding an other target type than Oracle Cloud Database is charged on monthly fee base as described here: Cloud Price List | Oracle

Cost and Usage Report

In the detailed cost and usage report, the target is marked as deleted (suffix DELETED + deletion date), and charged.

All you can do is getting angry about that mistake and wait. After a month, the money was burned down, and there were no more Oracle Cloud Infrastructure Data Safe costs charged. As you can see, there are 201.44 CHF charged for a month.

I don’t know what Oracle has for a currency converter, but actual 200 USD are less that 180 CHF 😉

Lessons learned

Pity about the beautiful money – and for my next test run: RTFM.

Oracle Cloud Infrastructure, Oracle Tools, Security, TrivadisContent