Streamlining Oracle Database Upgrades: Automated Blackouts with AutoUpgrade

AutoUpgrade is used to patch and upgrade Oracle database instances and today the “state-of-the-art” tool to bring Oracle database to a newer version. In a release from the this year 2025, a feature was introduced to automatically set a blackout in Oracle Enterprise Manager 13.5 onwards during the patching process. This removes the need for manual OEM blackout management steps to prevent downtime alerts during patching.

Technically it’s done by a local setup and usage of the Enterprise Manager CLI tool called emcli running on the database server. In the first AutoUpgrade release where this feature was announced, there were bugs in the tool as the blackout was set too late in OEM (which produced downtime alerts)  and the emcli command was not prepared with the correct parameters. Some mails later with AutoUpgrade’s Product Manager Daniel Overby-Hansen, I got a fix to test and it worked. I used the direct contact  to Daniel to speed up the process, had a testing environment up and running and we were in a close exchange until the problem was solved. Thank you Daniel for the uncomplicated way of solving the problem.

In the latest AutoUpgrade version downloadable at https://www.oracle.com/goto/upgrade or via MOS AutoUpgrade Tool (Doc ID 2485457.1), the bugs are fixed and settings blackouts works properly.

How to get the automated blackout running? This solution works for 13.5 and 24ai:

  • Setup User in Oracle Enterprise Manager
  • Install and configure command-line interface emcli
  • Define AutoUpgrade configuration parameters for database upgrade
  • Run AutoUpgrade

In this blog post, I show you the setup of emcli on an Oracle Linux 8 server working together with an Oracle Enterprise Manager 24ai.

Setup User in Oracle Enterprise Manager

This has not to be the SYSMAN, a user with less permissions is highly recommended. In this example, a new user called AUTOUPGRADE_USER is created with the tole EM_ALL_OPERATOR. This Role has privileges to manage Oracle Enterprise Manager operations like settings blackouts.

Install and configure command-line interface emcli

Install emcli as the OS user oracle. A JDK is required to setup emcli. You can either install a new JDK or use Java from an existing ORACLE_HOME installation. JDK download link: https://www.oracle.com/java/technologies/downloads.

Verify installed server Java version: (not that one from ORACLE_HOME/jdk):

$ java --version
java 23.0.2 2025-01-21
Java(TM) SE Runtime Environment (build 23.0.2+7-58)
Java HotSpot(TM) 64-Bit Server VM (build 23.0.2+7-58, mixed mode, sharing)

Create a new directory for the emcli software.

$ mkdir /u01/app/oracle/emcli
$ cd /u01/app/oracle/emcli

Download emcli from Enterprise Manager – replace the hostname with your OEM and port.

$ wget --no-check-certificate https://<YOUR-OEM-HOSTNAME>:7803/em/public_lib_download/emcli/kit/emclikit.jar

Install emcli package.

$ export JAVA_HOME=/usr/lib/jvm/jdk-23.0.2-oracle-x64
$ export PATH=$JAVA_HOME/bin:$PATH
$ $JAVA_HOME/bin/java -jar emclikit.jar -install_dir=/u01/app/oracle/emcli

Configure EMCLI with Autologin for user AUTOUPGRADE_USER.

$ ./emcli setup -url=https://<YOUR-OEM-HOSTNAME>:7803/em -username=autoupgrade_user -autologin -trustall

Verify emcli status.

$ ./emcli status
Oracle Enterprise Manager 24ai Release 1 EM CLI.
Copyright (c) 1996, 2024 Oracle Corporation and/or its affiliates. All rights reserved.

Instance Home          : /home/oracle/.emcli
Verb Jars Home         : /u01/app/oracle/emcli/./bindings/24.1.0.0.0/.emcli
Status                 : Configured
EM CLI Home            : /u01/app/oracle/emcli/.
EM CLI Version         : 24.1.0.0.0
Java Home              : /opt/jdk-17
Java Version           : 17.0.2
Log file               : /home/oracle/.emcli/.emcli.log
Log level              : SEVERE
EM URL                 : https://<YOUR-OEM-HOSTNAME>:7803/em
EM user                : autoupgrade_user
Auto login             : true
Trust all certificates : true

Define AutoUpgrade configuration parameters for database upgrade

This AutoUpgrade configuration file is for a release upgrade of a  single instance from Oracle 19.24 to 19.26 running on Oracle Linux. The AutoUpgrade jar-file is located in $ORACLE_BASE/autoupgrade.

  • Source: /u01/app/oracle/product/19.0.0/dbhome_1 is version 19.24
  • Target: /u01/app/oracle/product/19.0.0/dbhome_2 is version 19.24

The configuration file contains three lines to handle the Oracle Enterprise Manager blackout. You must define the path to emcli, a valid OEM target name and a suffix. This suffix help for a better identification in OEM during the blackout. The configuration file ORANGE.cfg:

# Global Settings
global.autoupg_log_dir=/u01/app/oracle/cfgtoollogs/autoupgrade
# Instance Settings
upg1.log_dir=/u01/app/oracle/cfgtoollogs/autoupgrade      # Path of the log directory
upg1.sid=ORANGE                                           # ORACLE_SID of the source DB/CDB
upg1.source_home=/u01/app/oracle/product/19.0.0/dbhome_1  # Path of the source ORACLE_HOME
upg1.target_home=/u01/app/oracle/product/19.0.0/dbhome_2  # Path of the target ORACLE_HOME
upg1.start_time=NOW                                       # Optional. [NOW | +XhYm (X hours, Y minutes after launch) | dd/mm/yyyy hh:mm:ss]
upg1.upgrade_node=dbengine02.internal                     # Optional. Run the hostname utility. Default is ''localhost''
upg1.emcli_path=/u01/app/oracle/emcli                     # EMCLI: Path to binary
upg1.em_target_name=ORANGE.internal                       # EMCLI: Target name in OEM
upg1.em_blackout_suffix=snc1724ol95                       # EMCLI: Suffix set for blackout identification

Run AutoUpgrade

AutoUpgrade and emcli doesn’t like the same Java version, but you can use the JDK from ORACLE_HOME instead. For other information about AutoUpgrade, check the documentation. AutoUpgrade is run as OS user oracle. Error when using JDK 23:

AutoUpgrade Analyze

$ export ORACLE_HOME=/u01/app/oracle/product/19.0.0/dbhome_1
$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -config ORANGE.cfg -mode analyze

AutoUpgrade Fixups

$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -config ORANGE.cfg -mode fixups

AutoUpgrade Upgrade

$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -config ORANGE.cfg -mode upgrade

During the upgrade process, a blackout is created in Oracle Enteprise Manager. With the suffix, the job can be identified.

Summary

This new AutoUpgrade functionality eliminates the manual operation to set and remove blackouts in Oracle Enterprise Manager during the upgrade process completely. I have already made an enhancement request to Daniel: The management of the OEM blackouts in 24ai via REST API, this would eliminate the installation of emcli locally. AutoUpgrade is THE upgrade tool, and it’s getting better and better each release.