EM13c Cloud Control

EM13c – Database Monitoring Template with DB Alert Log Metric

The new  DB Alert Log metric

Since the Oracle Enterprise Manager database plugin version 12.1.0.4, there is a new metric “DB Alert Log” available. This metric replaces the existing “Alert Log” metric. The benefit of the new plugin is if an ORA error is detected multiple times in the database alert log based on same error stack, it will be de-duplicated and only one metric alert is generated. Oracle’s recommendation is to replace this metric on the database targets – Changes to the Metric Alert Log Monitoring in Database Plug-in 12.1.0.4 and later (Doc ID 1587020.1).

How to create such metrics  and many more is very well described in Kellyn Pot’Vin blog post serie http://dbakevlar.com/2013/08/em12c-enterprise-monitoring-part-i/

The old metric Alert Log:

alert_log

The new metric DB Alert Log:

db_alert_log

In use the following thresholds for ORA- errors:

  • Critical alerts for ORA-600 and 7445
  • Warning alerts for all other ORA- errors

A basic Monitoring Template

To make my DBA job easier, I have created a basic template which monitors me the most important things. Sure, there are many more things what you can verify in EM13c, but I wanted to keep it simple. Feel free to extend your monitoring templates.

Metric Comparison Operator Warning Threshold Cricital Threshold Collection Schedule
Archive Area Archive Area Used (%) > 80 Every 15 Minutes
DB Alert Log Alert Log Error Trace File Every 5 Minutes
Alert Log Name
Archiver Hung Alert Log Error Contains ORA-
Data Block Corruption Alert Log Error Contains ORA-
Generic Alert Log Error Matches ORA-[0-9]+[^0-9] ORA-0*(600?|7445)[^0-9]
Media Failure Alert Log Error Contains ORA-
Session Terminated Alert Log Error Contains ORA-
Time/Line Number
Incident Generic Incident Matches .* Every 5 Minutes
Operational Error Data Block Corruption Matches .* Every 5 Minutes
Response Status = Down Every 15 Seconds
Tablespaces Full Tablespace Space Used (%) >= 85 97 Every 30 Minutes
Tablespaces Full (dictionary managed) Tablespace Space Used (%) (dictionary managed) >= 85 97 Every 30 Minutes

XML for Import

If you like to use this metric, you can download the XML here: db_monitoring_template_1.0  – just save the code as XML and import the metric in your Oracle EM13c Enterprise Manager.

Verification

After deploying of the template to the targets, verify on the target databases that a) the template is deployed and b) that the collection is enabled.Go to Oracle Database – Monitoring – Metric and Collection Settings (click on image for better view):

metrics

Useful My Oracle Support Notes

Oracle Enterprise Manager 13c – Tail your Logfiles

This are my favorite aliases for Enterprise Manager 13c troubleshooting. Just replace $ORACLE_BASE with your own installation directory and add these aliases in your .profile/.bash_profile.

Here is the URL to the Oracle documentation: http://docs.oracle.com/cd/E63000_01/EMADM/logging.htm#EMADM11796 – and feel free to create your own aliases e.g. for the *.OUT logfiles.

Oracle Management Server

NodeManager

Weblogic AdminServer

Weblogic EMGC_OMS1 Managed Server

Weblogic BIP Managed Server

OHS

Do you need a Mailserver for Testing Purposes? Communigate Pro!

When I am testing Oracle and other software, I want to verify functions like sending alert mails. On Linux systems it’s easy. The most distributions have packages like sendmail already preconfigured and ready to use. But for example on Windows systems, there is no built-in solution. The goal was to have a small virtual machine running as mailserver which can be used for all my test cases.

CommuniGate Pro

CommuniGate Pro is a groupware which supports a lot of solutions like SMTP, POP3, Webmail and many more. In this post I describe how you can easy install the software and configure the SMTP component in Oracle Enterprise Linux 6.

The goal is to use CommuniGate Pro for the Enterprise Manager 13c to send mail alerts. Note: If you want to use CommunigatePro in your company, then you have to license it.

Software Download

The Software is available for the most platforms here: http://www.communigate.com/main/purchase/download.html
Link to the installations guides: https://www.communigate.com/communigatepro/Install.html

Installation

I use the rpm file which I have already transferred to my server olten.trivadistraining.com into directory /tmp. The installation has to be done as user root.

The software is now installed in directory /opt

The installation routine creates the runlevel startup script that the software will be started automatically during server boot.

If you have already mail solutions like sendmail or postfix up and running, you have to stop and disable it.

Oracle Linux 6 – Stop postfix

Oracle Linux 6 –  Stop Sendmail

Start CommuniGate Pro

You can a) reboot your server or b) start it manually:

Web Access Configuration – Step by Step

Initial Configuration: https://www.communigate.com/communigatepro/Install.html#Config
Quick Start: https://www.communigate.com/communigatepro/QuickStart.html

When the server is started, you can now access the web GUI by server URL and Port 8010.

Step 1 – Master Password

Set master password for user postmaster – I agree.

communigate_01

Step 2 – Server Name and Interface Mode

Verify server name and language, set interface mode to Advanced –  Update.

communigate_02

Step 3 – Create Account

Tab Users – insert new account name –  Create Account.

communigate_03

Step 4  – Set Real Name and Password

Set the Real Name and the CommuniGate Password for the new created user – Update.

communigate_04

Step 5 – Add Client IP

In tab Settings – Network – Client IPs add your client ip address or address range – Update.

communigate_05

Configure Client – for Example Oracle Enterprise Manager 13c

Set Host, User Name and PasswordOK.

communigate_06

Now you will receive the messages from EM13c.

 

communigate_07

Troubleshooting

With telnet you can verify if the correct SMTP service is running on port 25 and if the firewall is open.

Wrong Service

Right Service

Summary

Communigate Pro offers you an easy to configure SMTP platform. I have configured a small virtual machine which I start whenever I want to test some email functions.

Sure, this software can do much more. But all I want is to send emails from my test servers – Mission accomplished!

Enterprise Manager 13c – Disable the BI Publisher

The BI Publisher will be started automatically during the startup process of the Enterprise Manager 13c. You don’t like the BI Publisher or you don’t use it? Save the resources, speed up your startup process, disable it. The password of the database repository owner SYSMAN is required.

Verify the Status – the BI Publisher is up and running

Disable the BI Publisher

Verify the Status again

If you want to enable the BI Publisher again, the command is listed below the emctl output.

Timing

I have tested the start of the Oracle Enterprise Manager in my local virtual machine environment (VMWare Workstation,15GB Memory, 4 Cores, SSD) with and without the BI Publisher. The difference:

BI Publisher enabled – 4 Minutes and 18 Seconds

BI Publisher disabled – 3 Minutes and 6 Seconds

Link to the Oracle Documentation

https://docs.oracle.com/cd/E63000_01/EMUPG/upgrading_oms.htm#CIHGBAJE

Summary

If you don’t use it – disable it.

Enterprise Manager 13c – Change Memory Settings with emctl

In Enterprise Manager Grid Control 11g, changes in the Oracle Management Server JVM memory settings could be done in a file called startEMServer.sh located in a domain subdirectory like /u00/app/oracle/product/gc_inst/user_projects/domains/GCDomain/bin/

And after an OMS restart, the JVM has used this new settings for the OMS.  Since 12c, it can be done by an emctl command.

emctl – Get your actual Settings OMS_HEAP_MAX

emctl – Change the Value

For example, the Memory will be resized to 2304M.

Verification

Restart your Oracle Management Server. In one of my tests this did not work. The OMS still used the old memory value. Another start/stop with emctl stop oms -all has worked.

The new value is now active:

In Linux, you can search for the java process with ps -ef | grep “java -server -Xms256M -Xmx”  – here are the first lines from my output where the Xmx parameter is located:

As you can see, the -Xmx Parameter is now set to 2304M, the Java virtual machine for EMGC_OMS1 is now using 2304M.

Link

Enterprise Manager Cloud Control Advanced Installation and Configuration Guide – Sizing Your Enterprise Manager Deployment