Oracle Release Update 19.9 – Lab Update Time (Grid Infrastructure Standalone & RDBMS)

The Oracle Release update 19.9 for Linux is available since a few days. Time to upgrade my lab environment at home which consists of the following components:

  • Oracle Grid Infrastructure Standalone 19.8.0 with ASM Normal Redundancy – +ASM
  • Oracle 19.8.0 RDBMS as Repository for Oracle Enterprise Manager – EMREPO

The running 19.8 Environment

Output from Trivadis base environment tool TVD-Basenv(TM).

oracle@mohnweg:/u01/app/19.0.0/grid/OPatch/ [em13cr4] u

TYPE (Cluster|DG) : SID/PROCESS       STATUS      HOME      [2020-10-25 19:47:47]
---------------------------------------------------------------------------------
Dummy em13cr3      : em13cr4           n/a         /u01/app/oracle/product/em13cr4
Dummy emagent      : emagent           n/a         /u01/app/oracle/product/agent13c/agent_13.4.0.0.0
Dummy emagent      : emagent_a         n/a         /u01/app/oracle/product/agent13c/agent_inst
Dummy grinf        : grinf19           n/a         /u01/app/19.0.0/grid
Dummy rdbms_ee     : rdbms19           n/a         /u01/app/oracle/product/19.0.0/dbhome_1

ASM-instance       : +ASM              up          /u01/app/19.0.0/grid
DB-instance (N|N)  : EMREPO            open        /u01/app/oracle/product/19.0.0/dbhome_1

Listener           : LISTENER          up          /u01/app/19.0.0/grid

Ohs 12c            :                   up          /u01/app/oracle/product/em13cr4/ohs
Agent              : emagent_a         up          /u01/app/oracle/product/agent13c/agent_inst

WLS admin          : EMGC_ADMINSERVER  up          /u01/app/oracle/product/em13cr4/wlserver
WLS mgmt           : EMGC_OMS1         up          /u01/app/oracle/product/em13cr4/wlserver
WLS node           : 1                 up          /u01/app/oracle/product/gc_inst/user_projects/domains/GCDomain/nodemanager

Oracle Restart
Process            : ocssd             up          /u01/app/19.0.0/grid
Process            : ohasd             up          /u01/app/19.0.0/grid

Patch Download, Transfer and Extract

I have downloaded the Combo which contains the RU for Grid Infrastructure and Oracle Java Virtual Machine.

  • COMBO OF OJVM RU COMPONENT 19.9.0.0.201020 + GI RU 19.9.0.0.201020(Patch 31720429)
    • OJVM RELEASE UPDATE 19.9.0.0.0(Patch 31668882)
    • GI RELEASE UPDATE 19.9.0.0.0(Patch 31750108)

The local stage directory with the extracted files:

oracle@mohnweg:/u01/app/oracle/stage/31720429/ [rdbms19] ll
total 32
drwxrwxrwx. 4 oracle oinstall    80 Oct 25 19:51 .
drwxr-xr-x. 3 oracle oinstall    63 Oct 25 19:51 ..
drwxr-xr-x. 4 oracle oinstall    67 Oct 25 19:51 31668882
drwxr-x---. 5 oracle oinstall    56 Oct 25 19:51 31750108
-rw-rw-r--. 1 oracle oinstall  4891 Oct 20 11:24 PatchSearch.xml
-rw-r--r--. 1 oracle oinstall 22562 Oct 20 17:55 README.html

OPatch

OPatch in Grid Infrastructure home directory has to be version 12.2.0.1.19 or later.

oracle@mohnweg:/u01/app/19.0.0/grid/OPatch/ [+ASM] ./opatch version
OPatch Version: 12.2.0.1.21

OPatch succeeded.

Version Verification

+ASM – Grid Infrastructure Standalone

oracle@mohnweg:~/ [+ASM] asmcmd -V
asmcmd version 19.8.0.0.0

ASMCMD> showversion
ASM version         : 19.8.0.0.0

EMREPO – RDBMS

oracle@mohnweg:~/ [EMREPO] sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Oct 25 19:44:39 2020
Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> select PATCH_ID,PATCH_TYPE,SOURCE_VERSION,TARGET_VERSION,STATUS from dba_registry_sqlpatch;

  PATCH_ID PATCH_TYPE SOURCE_VERSION  TARGET_VERSION  STATUS
---------- ---------- --------------- --------------- -------------------------
  31219897 INTERIM    19.1.0.0.0      19.1.0.0.0      SUCCESS
  31281355 RU         19.1.0.0.0      19.8.0.0.0      SUCCESS

CheckConflictAgainstOHWithDetail

+ASM – Grid Infrastructure Standalone

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31771877
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31772784
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31773437
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31780966

EMREPO – RDBMS

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31771877
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108/31772784

Take care, this line here produces an error:

oracle@mohnweg:~/ [+ASM] $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/app/oracle/stage/31720429/31750108
Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.0.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.0.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.0.0/grid/cfgtoollogs/opatch/opatch2020-10-25_20-00-57PM_1.log

This command doesn't support System Patch.

OPatch failed with error code 21

According My Oracle Support Note opatch CheckSystemSpace Command For Grid Infrastructure RU Fails With: “This command doesn’t support System Patch” (Doc ID 2634165.1), this error can be ignored and the line removed in future patch apply actions.

CheckSystemSpace

To check for space, we create there two files which contain the patch directories. The checks have to be successful.

+ASM – Grid Infrastructure Standalone

$ cat /tmp/patch_list_gihome.txt

/u01/app/oracle/stage/31720429/31750108/31771877
/u01/app/oracle/stage/31720429/31750108/31772784
/u01/app/oracle/stage/31720429/31750108/31773437
/u01/app/oracle/stage/31720429/31750108/31780966
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt

EMREPO – RDBMS

$ cat /tmp/patch_list_dbhome.txt
/u01/app/oracle/stage/31720429/31750108/31771877
/u01/app/oracle/stage/31720429/31750108/31772784
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt

Release Update Apply

As first action I stop the OEM. The I run opatchauto as user root. Grid Infrastructure and RBDMS components are stopped, started and patch automatically one by one. Here is the full output of the patch apply where you can see the executed steps. In my lab environment, it took about 20 minutes.

[root@mohnweg ~]# /u01/app/19.0.0/grid/OPatch/opatchauto apply /u01/app/oracle/stage/31720429/31750108

OPatchauto session is initiated at Sun Oct 25 20:21:31 2020

System initialization log file is /u01/app/19.0.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-10-25_08-21-34PM.log.

Session log file is /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/opatchauto2020-10-25_08-21-39PM.log
The id for this session is TERC

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0.0/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.0.0/dbhome_1

Verifying SQL patch applicability on home /u01/app/oracle/product/19.0.0/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.0.0/dbhome_1

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.0.0/grid
Patch applicability verified successfully on home /u01/app/19.0.0/grid

Preparing to bring down database service on home /u01/app/oracle/product/19.0.0/dbhome_1
Successfully prepared home /u01/app/oracle/product/19.0.0/dbhome_1 to bring down database service

Bringing down database service on home /u01/app/oracle/product/19.0.0/dbhome_1
Following database has been stopped and will be restarted later during the session: emrepo
Database service successfully brought down on home /u01/app/oracle/product/19.0.0/dbhome_1

Bringing down CRS service on home /u01/app/19.0.0/grid
Prepatch operation log file location: /u01/app/oracle/crsdata/mohnweg/crsconfig/hapatch_2020-10-25_08-24-02PM.log
CRS service brought down successfully on home /u01/app/19.0.0/grid

Start applying binary patch on home /u01/app/oracle/product/19.0.0/dbhome_1
Binary patch applied successfully on home /u01/app/oracle/product/19.0.0/dbhome_1

Start applying binary patch on home /u01/app/19.0.0/grid

Binary patch applied successfully on home /u01/app/19.0.0/grid

Starting CRS service on home /u01/app/19.0.0/grid
Postpatch operation log file location: /u01/app/oracle/crsdata/mohnweg/crsconfig/hapatch_2020-10-25_08-37-45PM.log
CRS service started successfully on home /u01/app/19.0.0/grid

Starting database service on home /u01/app/oracle/product/19.0.0/dbhome_1
Database service successfully started on home /u01/app/oracle/product/19.0.0/dbhome_1

Preparing home /u01/app/oracle/product/19.0.0/dbhome_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.0.0/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/19.0.0/dbhome_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:mohnweg
SIDB Home:/u01/app/oracle/product/19.0.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /u01/app/oracle/stage/31720429/31750108/31773437
Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /u01/app/oracle/stage/31720429/31750108/31780966
Reason: This patch is not applicable to this specified target type - "oracle_database"


==Following patches were SUCCESSFULLY applied:

Patch: /u01/app/oracle/stage/31720429/31750108/31771877
Log: /u01/app/oracle/product/19.0.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-24-30PM_1.log

Patch: /u01/app/oracle/stage/31720429/31750108/31772784
Log: /u01/app/oracle/product/19.0.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-24-30PM_1.log


Host:mohnweg
SIHA Home:/u01/app/19.0.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /u01/app/oracle/stage/31720429/31750108/31771877
Log: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-29-10PM_1.log

Patch: /u01/app/oracle/stage/31720429/31750108/31772784
Log: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-29-10PM_1.log

Patch: /u01/app/oracle/stage/31720429/31750108/31773437
Log: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-29-10PM_1.log

Patch: /u01/app/oracle/stage/31720429/31750108/31780966
Log: /u01/app/19.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-10-25_20-29-10PM_1.log

OPatchauto session completed at Sun Oct 25 20:41:38 2020
Time taken to complete the session 20 minutes, 7 seconds
[root@mohnweg ~]#

Version Verification

+ASM – Grid Infrastructure Standalone

oracle@mohnweg:~/ [+ASM] asmcmd -V
asmcmd version 19.9.0.0.0

ASMCMD> showversion
ASM version         : 19.9.0.0.0

EMREPO – RDBMS

oracle@mohnweg:~/ [EMREPO] sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Oct 25 19:44:39 2020
Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> select PATCH_ID,PATCH_TYPE,SOURCE_VERSION,TARGET_VERSION,STATUS from dba_registry_sqlpatch;

  PATCH_ID PATCH_TYPE SOURCE_VERSION  TARGET_VERSION  STATUS
---------- ---------- --------------- --------------- -------------------------
  31219897 INTERIM    19.1.0.0.0      19.1.0.0.0      SUCCESS
  31281355 RU         19.1.0.0.0      19.8.0.0.0      SUCCESS
  31771877 RU         19.8.0.0.0      19.9.0.0.0      SUCCESS

As you can see, the components were updated successfully. Time to start Oracle Enterprise Manager 13c Release 4 Update.

OJVM Apply

At the end, the OJVM patch has to applied. Set ORACLE_SID and ORACLE_HOME according the RDBMS environment.

Stop the RDBMS with srvctl

oracle@mohnweg:~/ [EMREPO] srvctl stop database -db emrepo

Change to OJVM Patch Directory

oracle@mohnweg:~/ [EMREPO] cd /u01/app/oracle/stage/31720429/31668882/

Apply the OJVM Patch

oracle@mohnweg:/u01/app/oracle/stage/31720429/31668882/ [EMREPO] $ORACLE_HOME/OPatch/opatch apply
Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/19.0.0/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/19.0.0/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.0.0/dbhome_1/cfgtoollogs/opatch/opatch2020-10-26_12-48-13PM_1.log

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   31668882

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/19.0.0/dbhome_1')


Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '31668882' to OH '/u01/app/oracle/product/19.0.0/dbhome_1'

Patching component oracle.javavm.server, 19.0.0.0.0...

Patching component oracle.javavm.client, 19.0.0.0.0...

Patching component oracle.javavm.server.core, 19.0.0.0.0...

Patching component oracle.rdbms.dbscripts, 19.0.0.0.0...

Patching component oracle.rdbms, 19.0.0.0.0...
Patch 31668882 successfully applied.
Sub-set patch [31219897] has become inactive due to the application of a super-set patch [31668882].
Please refer to Doc ID 2161861.1 for any possible further required actions.
Log file location: /u01/app/oracle/product/19.0.0/dbhome_1/cfgtoollogs/opatch/opatch2020-10-26_12-48-13PM_1.log

OPatch succeeded.

Startup Upgrade – Container Database and Pluggable Database – in SQL*Plus

oracle@mohnweg:~/ [EMREPO] sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Mon Oct 26 12:51:48 2020
Version 19.9.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup upgrade
ORACLE instance started.

Total System Global Area 5049940880 bytes
Fixed Size                  9145232 bytes
Variable Size            2248146944 bytes
Database Buffers         2785017856 bytes
Redo Buffers                7630848 bytes
Database mounted.
Database opened.
SQL> alter pluggable database all open upgrade;

Pluggable database altered.

Run datapatch

oracle@mohnweg:~/ [EMREPO] $ORACLE_HOME/OPatch/datapatch -verbose
SQL Patching tool version 19.9.0.0.0 Production on Mon Oct 26 12:53:30 2020
Copyright (c) 2012, 2020, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_2268_2020_10_26_12_53_30/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
Interim patch 31219897 (OJVM RELEASE UPDATE: 19.8.0.0.200714 (31219897)):
  Binary registry: Not installed
  PDB CDB$ROOT: Applied successfully on 13-SEP-20 12.47.47.620859 PM
  PDB PDB$SEED: Applied successfully on 13-SEP-20 12.47.50.075046 PM
  PDB PDBEMREPO01: Applied successfully on 13-SEP-20 12.47.50.075046 PM
Interim patch 31668882 (OJVM RELEASE UPDATE: 19.9.0.0.201020 (31668882)):
  Binary registry: Installed
  PDB CDB$ROOT: Not installed
  PDB PDB$SEED: Not installed
  PDB PDBEMREPO01: Not installed

Current state of release update SQL patches:
  Binary registry:
    19.9.0.0.0 Release_Update 200930183249: Installed
  PDB CDB$ROOT:
    Applied 19.9.0.0.0 Release_Update 200930183249 successfully on 25-OCT-20 08.41.35.152797 PM
  PDB PDB$SEED:
    Applied 19.9.0.0.0 Release_Update 200930183249 successfully on 25-OCT-20 08.41.35.369001 PM
  PDB PDBEMREPO01:
    Applied 19.9.0.0.0 Release_Update 200930183249 successfully on 25-OCT-20 08.41.35.578682 PM

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: CDB$ROOT PDB$SEED PDBEMREPO01
    The following interim patches will be rolled back:
      31219897 (OJVM RELEASE UPDATE: 19.8.0.0.200714 (31219897))
    No release update patches need to be installed
    The following interim patches will be applied:
      31668882 (OJVM RELEASE UPDATE: 19.9.0.0.201020 (31668882))

Installing patches...
Patch installation complete.  Total patches installed: 6

Validating logfiles...done
Patch 31219897 rollback (pdb CDB$ROOT): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31219897/23619699/31219897_rollback_EMREPO_CDBROOT_2020Oct26_12_54_11.log (no errors)
Patch 31668882 apply (pdb CDB$ROOT): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31668882/23790068/31668882_apply_EMREPO_CDBROOT_2020Oct26_12_54_11.log (no errors)
Patch 31219897 rollback (pdb PDB$SEED): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31219897/23619699/31219897_rollback_EMREPO_PDBSEED_2020Oct26_12_54_13.log (no errors)
Patch 31668882 apply (pdb PDB$SEED): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31668882/23790068/31668882_apply_EMREPO_PDBSEED_2020Oct26_12_54_13.log (no errors)
Patch 31219897 rollback (pdb PDBEMREPO01): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31219897/23619699/31219897_rollback_EMREPO_PDBEMREPO01_2020Oct26_12_54_13.log (no errors)
Patch 31668882 apply (pdb PDBEMREPO01): SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31668882/23790068/31668882_apply_EMREPO_PDBEMREPO01_2020Oct26_12_54_13.log (no errors)
SQL Patching tool complete on Mon Oct 26 12:54:16 2020

Shutdown Database in SQL*Plus

SQL> shutdown immediate

Start the RDBMS with srvctl

oracle@mohnweg:~/ [EMREPO] srvctl start database -db emrepo

Version Verification

EMREPO – RDBMS

oracle@mohnweg:~/ [EMREPO] sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Oct 25 19:44:39 2020
Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> select PATCH_ID,PATCH_TYPE,SOURCE_VERSION,TARGET_VERSION,STATUS,ACTION from dba_registry_sqlpatch;

  PATCH_ID PATCH_TYPE SOURCE_VERSION  TARGET_VERSION  STATUS                    ACTION
---------- ---------- --------------- --------------- ------------------------- ---------------
  31219897 INTERIM    19.1.0.0.0      19.1.0.0.0      SUCCESS                   APPLY
  31281355 RU         19.1.0.0.0      19.8.0.0.0      SUCCESS                   APPLY
  31771877 RU         19.8.0.0.0      19.9.0.0.0      SUCCESS                   APPLY
  31219897 INTERIM    19.9.0.0.0      19.9.0.0.0      SUCCESS                   ROLLBACK
  31668882 INTERIM    19.9.0.0.0      19.9.0.0.0      SUCCESS                   APPLY

Summary

There were no issues. Ok, it’s just a GI Standalone environment. But this was really a pleasure.