EM13c Cloud Control

Oracle Enterprise Manager EM13c – My Oracle Support is back!

Since a few weeks, the online connection in Oracle Enterprise Manager EM13c to My Oracle Support was broken. This resulted in errors like Error occurred when sending request to My Oracle Support, for example when you tried to set the My Oracle Support credentials in a fresh EM13c setup. Existing configurations were not able to get newest patch information anymore. Oracle has documented it in the note Recent Changes to MOS will Disable Enterprise Manager Online Connections (Doc ID 2664002.1)

Screenshot when trying to set MOS credentials in a new EM13c RU4 setup:

Now there is a fix available for all current EM 13 releases, RU included. For more details see Patch Now Available For Recent Changes to MOS That Disabled Enterprise Manager Online Connections (Doc ID 2678494.1). Here is the installation of the patch for an Oracle Enterprise Manager EM13c Release 4 RU2 without using a property file (optional). The patch is transferred to target OMS server and extracted in directory /u01/app/oracle/stage/31233849.

 

Stop OMS

Validate Patch Apply

Patch Apply

 

Start OMS

 

New try to set My Oracle Support Credentials – MOS is back!

Et voilà

MOS is back, take care if you use a proxy or a firewall – a new URL has to be whitelisted:  https://oauth-e.oracle.com !

Oracle Enterprise Manager 13c Release 4 – Time to Upgrade – First Experiences

The roll-out of the newest Oracle Enterprise Manager 13 Release 4 is a few days old, about time to try out the upgrade process in my ESXi lab environment.

First: This blog post about the OEM upgrade process is based on my own experience.

Software

https://www.oracle.com/enterprise-manager/downloads/cloud-control-downloads.html

Documents

https://docs.oracle.com/en/enterprise-manager/cloud-control/enterprise-manager-cloud-control/13.4/emupg/index.html

The Environment

  • Oracle Enterprise Manager 13.3 running on Oracle Linux 7.6 – OEM Patchlevel January 2020 – Non-HA
  • Oracle Enterprise Edition Repository 19.6.0 Single Tenant Database 
  • Oracle Restart / ASM 19.6.0
  • Additional Oracle Linux Server with 19.3.0 Container Databases
  • All targets up and running
  • My Oracle Support connected
  • Software staged directory in /u01/app/oracle/stage/em13cr4 and permission changed to execute bin file (chmod u+x em13400_linux64.bin).

Prerequisites

It’s important to verify the changed prerequisites – Chapter 3 – Prerequisites for Upgrading to Enterprise Manager Cloud Control 13c Release 4 – I had do disable all adaptive features in the repository pluggable database:

Restart the database after the changed settings.From my view this sentence here is wrong is the documentation (my database has version 19.6.0):

If your Management Repository is using Oracle Database 12.2 or higher, none of these parameters need to be set. 

Why? If these parameters are not set, the installer refuses working.

EMKEY Copy Requirements

The Key was copied to the repository. It well be removed after the successful upgrade.

Run Installer

Start the Oracle Universal Installer

The OMS is down now (emctl stop oms -all). Start the installer.

My Oracle Support Details

I am a registered user in My Oracle Support, I get enough information every week… I don’t enable this checkbox.

Software Updates

At the moment, there are no updates available. Maybe in future there will be any patches available for auto apply during the installation/upgrade process.

Installation Type

Upgrade an existing Enterprise Manager system, this one is my existing installation.

 

Installation Details

Enter a new Middleware Home Location.

Database Connection Details

Enter the SYS and SYSMAN password of the running repository. DDMP (Deferred Data Migration) Jobs are enabled. According the documentation, these jobs are running in the background when the OEM is starting up during the upgrade process to convert old data to the new format. If you have a huge amount of data in your earlier release, then the upgrade can take longer. You can run this job – if disabled – later as Post Upgrade Tasks.

Warnings

I will change to SHA communication later.

The repository has 19.6.0, all required patches are included.

I confirm that the JVMD engine is stopped, this has happened by stopping the OMS.

I had to stop the agent which was running on the management server.

Fix Parameter Settings by the Installer

For my environment, I let the installer fix this settings.

Plug-in Upgrade

Here we see the already installed plugins, they will be upgraded too.

Select Plug-ins

I don’t ant to install additional plug-ins.

Extend WebLogic Server Domain

Enter the password for the weblogic user and define the OMS Instance Base Location.

Enterprise Manager Shared Location Details

My Oracle Enterprise Manager doesn’t run in a high availability setup, I don’t need any shared location.

Port Configuration Details

I use the settings from the existing installation.

Review

Let’s start the upgrade!

Repository Upgrade failed

The upgrade process fails at the step where the repository has to be upgraded.

View Log – ORA-01950

When I scroll up the provided log from the installer, I see this error message here:

The schema manager logfile located in the 13.4 subdirectory $ORACLE_HOME/sysman/log/schemamanager confirms this error.

At this point, this is curious, because in 13.3, there were no objects of the SYSMAN in the USERS tablespace. All objects are located in OEM tablespaces with the MGMT prefix. Here is a list of objects in a repository where an Oracle Enterprise Manager 13c Release 3 is up and running.

The solution is simple. Just grant the permissions for the USERS tablespace to SYSMAN and retry the installation progress.But, don’t ask me why SYSMAN creates now objects in USERS…

After a while, you can see new created tables and indexes in tablespace USERS for user SYSMAN.

The repository upgrade step runs fine now, the whole upgrade process continuous.

Finish

After while, yes we did it, the root script execution is the last task. 

Script execution.

The Enterprise Manager is now running with Release 4.

About Enterprise Manager

Next steps are

  • upgrade the Oracle agents
  • uninstall the old OEM software –  btw, the installer already detaches the old ORACLE_HOME from Oracle’s central inventory for you and removes the emkey from the repository 🙂

Summary

This is a lab environment, not a huge setup with hundreds of targets. But we can see here two important points for the upgrade process. a) read the manual and disable all adaptive features, even when you have an 19c database and b) user SYSMAN needs permission on the USERS tablespace. This is very unusual and should be corrected from my side. But now, enjoy Oracle Enterprise Manager 13c Release 4!

OPatch 13.9 in EM13c – Say Goodbye to Unzip, Copy & Paste

Yesterday I wanted to apply a brand new patch to customer’s Enterprise Manager 13cR2 OMS running on Linux. First I updated the OMSPatcher as described here: How to upgrade the 13.1 Cloud Control OMSPatcher to latest version of OMSPatcher (Doc ID 2135028.1). This update was easy. Download, transfer, unzip and copy the OMSPatcher files to the Enterprise Manager ORACLE_HOME directory.

Then the patch apply results in an error. OMSPatcher is based on OPatch, he needs an update too.

We need a new OPatch Version

This was error message for patch apply with omspatcher when I executed the apply command.

My actual EM13cR2 OPatch version was 13.8.0.0.0.

Get the new Version

In My Oracle Support I found the newest version of OPatch, Download.

opatch_01

Transfer and extract

The next step was to extract and to transfer the package to the target server.And then I was wondering that not like in versions before an OPatch directory was created, now there is a JAR file inside the package. Copy & Paste of an OPatch directory is no more possible anymore.

Installation Routine – README.txt

Let’s take a look in the README.txt – there is an installation manual.

Installation – 1st run – where is my JDK ?

Installation – 2nd run – JRE is not good enough

I tried to use the existing JDK from my Oracle 12.1 installation

Installation – 3rd run – JDK 1.7 rocks

Now I have downloaded and configured JDK 1.7. Another try, and now it works. Here is a short summary of the output. In the background now the Oracle Universal Installer is started.

Thanks to Gokhan Atil – http://www.gokhanatil.com – for the second possibility, you can use the JDK from the OMS too.

Job done

Now the version was ok, the patch could be applied.

Additional Information: If the installer does not find your inventory location (was happened to me on an AIX system), just execute the install command with the parameter -invPtrLoc.

The OMSPatcher can be updated like before, but the other important component in the patch process OPatch has has to be installed. From my point of view this is a step back, I don’t know what was the idea behind to change it. Now you have to be sure, that there is an existing JDK 1.7 available on your system. This means that applying a patch on the express way is not possible anymore, you have to fulfill the prerequisite before you have an actual OPatch. And at the moment, there is no other documentation than the README.txt available, the hint with the inventory parameter came from My Oracle Support after I opened a SR.

Say good-bye to Unzip, Copy & Paste.