Oracle Cloud Infrastructure

Oracle Cloud Infrastructure – A short Blog Post about a secure and small Development Setup

Author: Martin Berger 14. January 2021

For an internal project I had the pleasure to setup a new Oracle Cloud Infrastructure environment for an APEX development team. Here is a short overview about the setup.

Requirements

VPN Access from everywhere – 2 people are working maximal at same time on the environment
Oracle Standard Edition 2 – no license available in project
Small monitoring to verify server stats
Instances can be started and stopped from the developers to save costs for example over night, weekend, holiday etc.

Architecture Diagram

Resource	Network	Usage	Remarks
Open VPN Access Server	Public Subnet	VPN client access and traffic routing	OCI Cloud Marketplace Image – OpenVPN Access Server (2 FREE VPN Connections) – OpenVPN Inc. – Oracle Cloud Marketplace
Management Server	Private Subnet	OCI-CLI, Monitoring	Application server and database node start/stop with OCI-CLI, Grafana and Prometheus for monitoring
Application Server	Private Subnet	Tomcat ORDS, APEX	–
Database System	Private Subnet	–	OCI Database Standard Edition 2, Backup to Object Store enabled

Network Components

Regional private and public subnet
Security lists and network security groups
Private and public routing table
NAT gateway for regional private subnet

Monitoring

Grafana and Prometheus, running on the management server. The free shape VM.Standard.E2.1.Micro fits perfect for this small setup! The Prometheus node exporter runs on the database and the application server. I used this Grafana dashboard here: Prometheus Node Exporter Full dashboard for Grafana | Grafana Labs

Next Steps

Adding Influx DB for persistence
Adding the Oracle database to Grafana monitoring
Optimizing shape size for the database server according usage

Other Ideas

Create a blueprint for internal developer environments
Automate the setup with Terraform and Ansible

Summary

Setting up this infrastructure in Oracle Cloud Infrastructure was fun. All developer requirements are fulfilled. Started with the Network and OpenVPN configuration – I really like their Marketplace instance – and the moved on to application and database server, step-by-step. There are many other ideas what we can do more based on this setup, the work will not run out. #ilike

Oracle Cloud Infrastructure, Oracle Tools, Security, Testing, TrivadisContent

Oracle Cloud Infrastructure Data Safe – How to burn down 201.44 Swiss Francs in 30 Seconds…

Author: Martin Berger 11. January 2021

Is Data Safe really for free?

In the last autumn, the new Oracle Cloud Infrastructure feature called Data Safe was released. For sure, new features has to be tested. I have tested the Data Safe feature too and added a cloud database to Data Safe. But in my enthusiasm about this cool feature – or maybe it was just too late in the evening – I did a mistake by adding the database target. Four days later, I recognized that Data Safe is charged in my account. Mmm, but should it not be for free? First reaction: I raised an SR and described the case. The nice guy from My Oracle Support realized the situation quickly:

Dear Mister Berger, you have used the wrong target type when adding the Oracle Cloud Infrastructure database as a new Data Safe target.

From the Service Request:

B91632 – Oracle Cloud Infrastructure – Data Safe for Database Cloud Service – Each (Includes 1 million audit records per target per month) – Free
B91631 – Oracle Cloud Infrastructure – Data Safe for Database Cloud Service – Audit Record Collection Over 1 Million Records (over 1 million audit records per target per month) – 0.0800 / 10,000 Audit Records Per Target Per Month
B92733 – Oracle Cloud Infrastructure – Data Safe for On-Premises Databases – Target Database Per Month – 200.00 Target Database Per Month + Includes 1 million audit records per target per month (pre-requisite under B91632)

Indeed, indeed. According My Oracle Support I have used the wrong target type. Instead Oracle Cloud Database, I used Oracle Database on Compute. And did not realized, the mistake and ignored the text below to the dropdown box. Shame on me 😉 – here is the small, but important difference:

So far so good, the mistake was recognized. I deleted the target and added it from scratch with the correct target type. But this didn’t help, the charging went on.

Oracle Cloud Infrastructure Price List

Adding an other target type than Oracle Cloud Database is charged on monthly fee base as described here: Cloud Price List | Oracle

Cost and Usage Report

In the detailed cost and usage report, the target is marked as deleted (suffix DELETED + deletion date), and charged.

All you can do is getting angry about that mistake and wait. After a month, the money was burned down, and there were no more Oracle Cloud Infrastructure Data Safe costs charged. As you can see, there are 201.44 CHF charged for a month.

I don’t know what Oracle has for a currency converter, but actual 200 USD are less that 180 CHF 😉

Lessons learned

Pity about the beautiful money – and for my next test run: RTFM.

Oracle Cloud Infrastructure, Oracle Tools, Security, TrivadisContent

Oracle OCI Data Transfer Service – A journey from Kestenholz/Jurasüdfuss/Switzerland to Frankfurt and back

Author: Martin Berger 24. November 2020

The Oracle Data Transfer service is a offline data transfer method to migrate data to the Oracle Cloud Infrastructure. A transfer service is useful, when your network bandwidth and connection is not sufficient to upload your migration data in a meaningful time. Oracle offers two methods: The disk-based data transfer and the appliance-based data transfer. The service is no only one-way, data can also be exported in an Oracle Cloud Infrastructure data center and shipped to your data center.

According one of my Company Trivadis’ cultural value called curiosity, I was wondering how this service works. This is the story of a tiny USB Hard Disk Drive full of data, which was going on a long journey from Kestenholz / Jurasüdfuss / Solothurn / Switzerland to the Oracle Infrastructure data center in Frankfurt and back.

Setup

The OCI Data Transfer utility is Linux based, the USB 3.0 HDD is attached to a VMware Virtual machine where Oracle Linux is running
The virtual machine has access to the Internet
Data is available – for this example I used some open data from Swiss government (opendata.swiss)

Data Transfer Service Regions

Actually data transfer is available in Frankfurt, Ashburn, Phoenix, London and Osaka. From Switzerland, Frankfurt is the nearest location.

How is your data coming into the Oracle Cloud

Enable Data Transfer Service – Data Transfer
Prepare an Object Storage bucket
Create a Transfer Job
Attach a HDD to a Linux based host, use the Data Transfer Utility to create and encrypt the device
Copy data to the HDD
Create and upload the disk manifest
Lock the disk
Attach the Disk to the transfer label and create the package
Create a transfer package
Shipping and update shipping information
Tracking
Data Verification
Object Storage Replication Policy (optional)
Finally…

Note: Most of the jobs above can be done by the OCI CLI on command line and are very well described in the Oracle doumentation.

1. Enable Data Transfer Service – Entitlement

Before you can use this service, the Data Transfer service has to be enabled in general. Therefore you have to request it. The OCI tenant administrator gets a document, what he has to sign in a digital way. It contains for example a description how to bring data to OCI, and if you order an appliance that there will be a 45day maximum limit where the appliance has to be returned to Oracle. And a few days later, the service is ready to use. Basically now you have the permissions, to order a Data Transfer Appliance, but in this test I used the Disk Service.

2. Prepare an Object Storage Bucket

In the Frankfurt region, I created a new Object Storage bucket called data_transfer_usb. This is the bucket where the shipped data will be transferred in.

3. Create a Transfer Job

In Object Storage – Data Transfer Import, we create a new transfer job. It contains the upload bucket from above and as method the transfer device type is disk. For furtehr processing, we need the OCID of the job. As you can see, actually there is no transfer disk attached.

4. Attach a HDD to a Linux based host, use the Data Transfer Utility to create and encrypt the Device

Prerequisites for the Data Transfer Utility according the documentation:

An OCI Account which have the IAM permissions for Data Transfer
A Linux machine with Oracle Linux 6 or greater, Ubuntu 14.04 or greater, SUSE 11 or greater
Java 1.8 or 1.11
hdparm 9.0 or later
Cryptsetup 1.2.0 or later

Package Installation for my Oracle Linux 7 Machine

# yum install java-1.8.0-openjdk-devel
# yum install hdparm
# yum install cryptsetup

# yum install java-1.8.0-openjdk-devel

# yum install hdparm

# yum install cryptsetup

Download and Installation of the Data Transfer Utility

The actual link to the file is in the online documentation.

# wget https://docs.cloud.oracle.com/tools/dts-rpm/3.0.2461/dts-3.0.2461.x86_64.rpm
# yum localinstall /tmp/dts-3.0.2461.x86_64.rpm

1 2	# wget https://docs.cloud.oracle.com/tools/dts-rpm/3.0.2461/dts-3.0.2461.x86_64.rpm # yum localinstall /tmp/dts-3.0.2461.x86_64.rpm

Test

# dts --version
3.0.2461 ( DEV_NO_HASH )

1 2	# dts --version 3.0.2461 ( DEV_NO_HASH )

Configure IAM Credentials for Data Transfer Actions

The configuration is according configuring the Oracle Cloud Infrastructure CLI with user, fingerprint, key_file, tenancy and region. Example configuration file:

[DEFAULT]
user=ocid1.user.oc1..aaaaaaaadlv5vwm3zfdnyanosmdjm7cyjzdhmgawre12345671234567
fingerprint=6a:ce:8e:a7:4a:a1:70:ad:4e:d7:e6:8f:d1:23:45:g6
key_file=/root/.oci/oci_api_key.pem
tenancy=ocid1.tenancy.oc1..aaaaaaaaxuk4je4t3aorovuzmwyeaq5sftqv3nkyz64s12345671234567
region=eu-frankfurt-1

[DEFAULT]

user=ocid1.user.oc1..aaaaaaaadlv5vwm3zfdnyanosmdjm7cyjzdhmgawre12345671234567

fingerprint=6a:ce:8e:a7:4a:a1:70:ad:4e:d7:e6:8f:d1:23:45:g6

key_file=/root/.oci/oci_api_key.pem

tenancy=ocid1.tenancy.oc1..aaaaaaaaxuk4je4t3aorovuzmwyeaq5sftqv3nkyz64s12345671234567

region=eu-frankfurt-1

Verify Credentials

# dts job verify-upload-user-credentials --bucket data_transfer_usb
created object BulkDataTransferTestObject in bucket data_transfer_usb
overwrote object BulkDataTransferTestObject in bucket data_transfer_usb
inspected object BulkDataTransferTestObject in bucket data_transfer_usb
read object BulkDataTransferTestObject in bucket data_transfer_usb

# dts job verify-upload-user-credentials --bucket data_transfer_usb

created object BulkDataTransferTestObject in bucket data_transfer_usb

overwrote object BulkDataTransferTestObject in bucket data_transfer_usb

inspected object BulkDataTransferTestObject in bucket data_transfer_usb

read object BulkDataTransferTestObject in bucket data_transfer_usb

Show Data Transfer Job Details – Status is PREPARED

Here you can see the shipping address from the Oracle Infrastructure data center frankfurk and the label. Both information are used later in process.

# dts job show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekvpmgdnnobcgrdb4awa7c5q
Transfer Job :
ID : ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekvpmgdnnobcgrdb4awa7c5q
CompartmentId : ocid1.compartment.oc1..aaaaaaaat5uo2xh77edws4huwvqorengp7x4xdv6x3giw3vryk36vyydwsdq
UploadBucket : data_transfer_usb
Name : data_transfer_usb_job_01
Label : J5Y0R5KAS
CreationDate : 2020/10/27 12:53:02 CET
Status : PREPARING
freeformTags : *** none ***
definedTags :
Oracle-Tags :
CreatedBy : oracleidentitycloudservice/martin.berger@trivadis.com
CreatedOn : 2020-10-27T11:53:02.320Z
Packages :
[1] :
Label : PDQ0QMLNT
TransferSiteShippingAddress : Telehouse c/o Oracle Data Transfer Service; Christos Panoudis; Phone: +49152 22882673 Job:J5Y0R5KAS Package:PDQ0QMLNT ; Kleyerstraße 75-87; 60326 Frankfurt am Main; Germany
DeliveryVendor : DHL
DeliveryTrackingNumber : *** none ***
ReturnDeliveryTrackingNumber : *** none ***
Status : PREPARING
Devices : [*** none ***]
UnattachedDevices : [*** none ***]
Appliances : [*** none ***]

# dts job show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekvpmgdnnobcgrdb4awa7c5q

Transfer Job :

ID : ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekvpmgdnnobcgrdb4awa7c5q

CompartmentId : ocid1.compartment.oc1..aaaaaaaat5uo2xh77edws4huwvqorengp7x4xdv6x3giw3vryk36vyydwsdq

UploadBucket : data_transfer_usb

Name : data_transfer_usb_job_01

Label : J5Y0R5KAS

CreationDate : 2020/10/27 12:53:02 CET

Status : PREPARING

freeformTags : *** none ***

definedTags :

Oracle-Tags :

CreatedBy : oracleidentitycloudservice/martin.berger@trivadis.com

CreatedOn : 2020-10-27T11:53:02.320Z

Packages :

[1] :

Label : PDQ0QMLNT

TransferSiteShippingAddress : Telehouse c/o Oracle Data Transfer Service; Christos Panoudis; Phone: +49152 22882673 Job:J5Y0R5KAS Package:PDQ0QMLNT ; Kleyerstraße 75-87; 60326 Frankfurt am Main; Germany

DeliveryVendor : DHL

DeliveryTrackingNumber : *** none ***

ReturnDeliveryTrackingNumber : *** none ***

Status : PREPARING

Devices : [*** none ***]

UnattachedDevices : [*** none ***]

Appliances : [*** none ***]

Prepare USB Hard Disk Drive

The disk is attached as /dev/sdb – it is a Western Digital drive. Important: The disk needs no partition.

# fdisk -l /dev/sdb
Disk /dev/sdb: 931.5 GiB, 1000170586112 bytes, 1953458176 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe7ec0c10

Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 1953458175 1953456128 931.5G 7 HPFS/NTFS/exFAT

# hdparm -I /dev/sdb | grep Model
Model Number: WDC WD10JMVW-11S5XS0

# fdisk -l /dev/sdb

Disk /dev/sdb: 931.5 GiB, 1000170586112 bytes, 1953458176 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: dos

Disk identifier: 0xe7ec0c10

Device Boot Start End Sectors Size Id Type

/dev/sdb1 2048 1953458175 1953456128 931.5G 7 HPFS/NTFS/exFAT

# hdparm -I /dev/sdb | grep Model

Model Number: WDC WD10JMVW-11S5XS0

Create Transfer Disk for Data Copy

This command will setup the disk and mount it immediately. As additional information we need the disk label for further processing.

# dts disk create --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --block-device /dev/sdb

Will prepare /dev/sdb
serial number WD-WX61E82FY633 as an encrypted block device.
=> Set up disk for encryption using cryptsetup.
=> Create file system.
=> Mount file system.

Continue?? [y/n]y

---> IMPORTANT: RECORD THE ENCRYPTION PASSPHRASE: [m, o, 3, 4, +, j, w, I, 3, +, s, M, =] <---


The passphrase cannot be retrieved again. It will be used to encrypt this disk
and normally will not be needed again. However, if the system is restarted before
all files are copied to the filesystem and the disk is then finalized through this
CLI, you will need to provide the passphrase.

Continue?? [y/n]y

The disk label is DAMOED7GH
DO NOT INTERRUPT THIS COMMAND WHILE THE DISK IS BEING PREPARED.
This can take several minutes ...

initialized device for encryption
set up device for encryption
creating ext4 filesystem on /dev/mapper/WD-WX61E82FY633 ...
Found a atari partition table in /dev/mapper/WD-WX61E82FY633
created filesystem

Successfully prepared the disk.
You can now copy files to the file system.
After all files are copied, run this command to create the manifest:

dts disk manifest --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH [--object-name-prefix <object_name_prefix>]

Transfer Disk :
Label : DAMOED7GH
SerialNumber : WD-WX61E82FY633
Status : PREPARING
EncryptionPassphrase : mo34+jwI3+sUaL/NORhXOKQU9k12345671234567

# dts disk create --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --block-device /dev/sdb

Will prepare /dev/sdb

serial number WD-WX61E82FY633 as an encrypted block device.

=> Set up disk for encryption using cryptsetup.

=> Create file system.

=> Mount file system.

Continue?? [y/n]y

---> IMPORTANT: RECORD THE ENCRYPTION PASSPHRASE: [m, o, 3, 4, +, j, w, I, 3, +, s, M, =] <---

The passphrase cannot be retrieved again. It will be used to encrypt this disk

and normally will not be needed again. However, if the system is restarted before

all files are copied to the filesystem and the disk is then finalized through this

CLI, you will need to provide the passphrase.

Continue?? [y/n]y

The disk label is DAMOED7GH

DO NOT INTERRUPT THIS COMMAND WHILE THE DISK IS BEING PREPARED.

This can take several minutes ...

initialized device for encryption

set up device for encryption

creating ext4 filesystem on /dev/mapper/WD-WX61E82FY633 ...

Found a atari partition table in /dev/mapper/WD-WX61E82FY633

created filesystem

Successfully prepared the disk.

You can now copy files to the file system.

After all files are copied, run this command to create the manifest:

dts disk manifest --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH [--object-name-prefix <object_name_prefix>]

Transfer Disk :

Label : DAMOED7GH

SerialNumber : WD-WX61E82FY633

Status : PREPARING

EncryptionPassphrase : mo34+jwI3+sUaL/NORhXOKQU9k12345671234567

Mount point is /mnt/orcdts_DAMOED7GH.

# df -TH /mnt/orcdts_DAMOED7GH
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/WD-WX61E82FY633 ext4 984G 80M 934G 1% /mnt/orcdts_DAMOED7GH

# df -TH /mnt/orcdts_DAMOED7GH

Filesystem Type Size Used Avail Use% Mounted on

/dev/mapper/WD-WX61E82FY633 ext4 984G 80M 934G 1% /mnt/orcdts_DAMOED7GH

The Transfer Disk status has changed to PREPARING and the disk serial number is registered now.

5. Copy Data to HDD

For the test run I have copied some Open Data stuff, an Oracle Backup and Oracle Data Pump export files to the disk.

# ll /mnt/orcdts_DAMOED7GH
total 1540788
-rw-r--r--. 1 root root 13816663 Oct 28 11:07 basel-opendata.tgz
-rw-r--r--. 1 root root 132009 Oct 28 11:07 bernmobil-opendata.tgz
-rw-r--r--. 1 root root 28493853 Oct 28 11:07 bundesverwaltung-opendata.tgz
-rw-r--r--. 1 root root 307998059 Oct 28 11:07 expdphr-28-10-20-01.dmp.tgz
-rw-r--r--. 1 root root 307998059 Oct 28 11:07 expdphr-28-10-20-02.dmp.tgz
drwx------. 2 root root 16384 Oct 28 11:03 lost+found
-rw-r--r--. 1 root root 695625592 Oct 28 11:08 RMAN_CDB118_28102020.tgz
-rw-r--r--. 1 root root 223666017 Oct 28 11:08 sbb-opendata.tgz

# ll /mnt/orcdts_DAMOED7GH

total 1540788

-rw-r--r--. 1 root root 13816663 Oct 28 11:07 basel-opendata.tgz

-rw-r--r--. 1 root root 132009 Oct 28 11:07 bernmobil-opendata.tgz

-rw-r--r--. 1 root root 28493853 Oct 28 11:07 bundesverwaltung-opendata.tgz

-rw-r--r--. 1 root root 307998059 Oct 28 11:07 expdphr-28-10-20-01.dmp.tgz

-rw-r--r--. 1 root root 307998059 Oct 28 11:07 expdphr-28-10-20-02.dmp.tgz

drwx------. 2 root root 16384 Oct 28 11:03 lost+found

-rw-r--r--. 1 root root 695625592 Oct 28 11:08 RMAN_CDB118_28102020.tgz

-rw-r--r--. 1 root root 223666017 Oct 28 11:08 sbb-opendata.tgz

6. Generate Manifest File

It generates a local file which contains a list of the files and her MD5 checksums like an inventory file. Here the disk label is required.

# dts disk manifest --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH
Walking the filesystem to gather stats and to check for errors
Reading all checkpoint data. This might take a while depending on checkpoint log size.
2020/10/28 11:09:18 Manifest generation started

Files processed : 2 / 8
Bytes processed : 41.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 103.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 164.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 228.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 284.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 353.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 412.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 476.15 MB / 1.47 GB

Files processed : 2 / 8
Bytes processed : 545.15 MB / 1.47 GB

Files processed : 3 / 8
Bytes processed : 616.88 MB / 1.47 GB

Files processed : 4 / 8
Bytes processed : 881.06 MB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 950.54 MB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1007.54 MB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.04 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.11 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.18 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.25 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.31 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.37 GB / 1.47 GB

Files processed : 6 / 8
Bytes processed : 1.41 GB / 1.47 GB

Files processed : 8 / 8
Bytes processed : 1.47 GB / 1.47 GB

# dts disk manifest --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH

Walking the filesystem to gather stats and to check for errors

Reading all checkpoint data. This might take a while depending on checkpoint log size.

2020/10/28 11:09:18 Manifest generation started

Files processed : 2 / 8

Bytes processed : 41.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 103.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 164.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 228.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 284.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 353.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 412.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 476.15 MB / 1.47 GB

Files processed : 2 / 8

Bytes processed : 545.15 MB / 1.47 GB

Files processed : 3 / 8

Bytes processed : 616.88 MB / 1.47 GB

Files processed : 4 / 8

Bytes processed : 881.06 MB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 950.54 MB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1007.54 MB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.04 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.11 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.18 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.25 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.31 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.37 GB / 1.47 GB

Files processed : 6 / 8

Bytes processed : 1.41 GB / 1.47 GB

Files processed : 8 / 8

Bytes processed : 1.47 GB / 1.47 GB

The file:

# cat /mnt/orcdts_DAMOED7GH/.bulk_data_transfer/metadata/manifest

{"path":"logFile.log","md5":"PZdMVbAydkbIo3JuJW5HKQ==","lastModifiedTime":1603879758896,"size":26710}
{"path":"bernmobil-opendata.tgz","md5":"J6VvwZemr5ayF+hZmEMvUQ==","lastModifiedTime":1603879648205,"size":132009}
{"path":"expdphr-28-10-20-02.dmp.tgz","md5":"tgtoAyh79EaAHnG2cMhJ4g==","lastModifiedTime":1603879668772,"size":307998059}
{"path":"basel-opendata.tgz","md5":"rRKvfnfw3kjRgp+yyJd9eg==","lastModifiedTime":1603879648180,"size":13816663}
{"path":"sbb-opendata.tgz","md5":"iggsOeCuvpE+zDlwoYmXcA==","lastModifiedTime":1603879699757,"size":223666017}
{"path":"bundesverwaltung-opendata.tgz","md5":"Pbs8W5rNpqs+x48bb/wVQw==","lastModifiedTime":1603879649038,"size":28493853}
{"path":"RMAN_CDB118_28102020.tgz","md5":"6ivSfbVKHceQ6FMy2xVSNA==","lastModifiedTime":1603879692329,"size":695625592}
{"path":"expdphr-28-10-20-01.dmp.tgz","md5":"tgtoAyh79EaAHnG2cMhJ4g==","lastModifiedTime":1603879658104,"size":307998059}

# cat /mnt/orcdts_DAMOED7GH/.bulk_data_transfer/metadata/manifest

{"path":"logFile.log","md5":"PZdMVbAydkbIo3JuJW5HKQ==","lastModifiedTime":1603879758896,"size":26710}

{"path":"bernmobil-opendata.tgz","md5":"J6VvwZemr5ayF+hZmEMvUQ==","lastModifiedTime":1603879648205,"size":132009}

{"path":"expdphr-28-10-20-02.dmp.tgz","md5":"tgtoAyh79EaAHnG2cMhJ4g==","lastModifiedTime":1603879668772,"size":307998059}

{"path":"basel-opendata.tgz","md5":"rRKvfnfw3kjRgp+yyJd9eg==","lastModifiedTime":1603879648180,"size":13816663}

{"path":"sbb-opendata.tgz","md5":"iggsOeCuvpE+zDlwoYmXcA==","lastModifiedTime":1603879699757,"size":223666017}

{"path":"bundesverwaltung-opendata.tgz","md5":"Pbs8W5rNpqs+x48bb/wVQw==","lastModifiedTime":1603879649038,"size":28493853}

{"path":"RMAN_CDB118_28102020.tgz","md5":"6ivSfbVKHceQ6FMy2xVSNA==","lastModifiedTime":1603879692329,"size":695625592}

{"path":"expdphr-28-10-20-01.dmp.tgz","md5":"tgtoAyh79EaAHnG2cMhJ4g==","lastModifiedTime":1603879658104,"size":307998059}

7. Lock the Disk

# dts disk lock --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH --block-device /dev/sdb
Copying upload user credentials.
created object BulkDataTransferTestObject in bucket data_transfer_usb
overwrote object BulkDataTransferTestObject in bucket data_transfer_usb
inspected object BulkDataTransferTestObject in bucket data_transfer_usb
read object BulkDataTransferTestObject in bucket data_transfer_usb
Scanning filesystem to validate manifest. If special files are encountered, they will be listed below.
validated manifest
/dev/sdb DAMOED7GH is encrypted and locked
Locked disk.

# dts disk lock --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH --block-device /dev/sdb

Copying upload user credentials.

created object BulkDataTransferTestObject in bucket data_transfer_usb

overwrote object BulkDataTransferTestObject in bucket data_transfer_usb

inspected object BulkDataTransferTestObject in bucket data_transfer_usb

read object BulkDataTransferTestObject in bucket data_transfer_usb

Scanning filesystem to validate manifest. If special files are encountered, they will be listed below.

validated manifest

/dev/sdb DAMOED7GH is encrypted and locked

Locked disk.

8. Attach the Disk to the Transfer Label and create the Package

The status now changes to ACTIVE.

# dts disk attach --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH --package-label PDQ0QMLNT
Attached disk: DAMOED7GH to package: PDQ0QMLNT

1 2	# dts disk attach --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --disk-label DAMOED7GH --package-label PDQ0QMLNT Attached disk: DAMOED7GH to package: PDQ0QMLNT

9. Shipping and Shipping Information Update

As shipping company I used DHL Switzerland. They have a pick point near by in Langenthal. At this point, it’s important to organize the return shipping too and put the return shipping label in the box. I didn’t realize it and have forgotten to organize the return shipping. So the disk was stranded in the Frankfurt data center. And then the story began. As a private person, the delivery companies DHL and UPS doesn’t allow private persons to re-import packages from outside Switzerland without a customer number. But, private persons don’t get such a number. Finally with FedEx I was able to organize the return shipping. Thanks to Andrew and Christos from Oracle’s OCI Data Transfer team for their patience!

Note: Companies like DHL have templates to create pro-forma commercial invoices – https://www.dhl.ch/exp-de/express/zollabwicklung/zollpapiere/proforma_rechnung.html#invoice

This disk was sent to the Oracle Cloud Infrastructure data center Frankfurt.

Now the shipping information has to be updated with vendor and the tracking numbers.

# dts package ship --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT --package-vendor DHL --tracking-number 5542918614 --return-tracking-number 12345678
Update package: PDQ0QMLNT

1 2	# dts package ship --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT --package-vendor DHL --tracking-number 5542918614 --return-tracking-number 12345678 Update package: PDQ0QMLNT

10. Tracking

DHL required two days until delivery in Frankfurt. Oracle started one day later with the data import.

11. Data Processing

Then Oracle is uploading the data and the disk is attached, the job transfer status changes to PROCESSING.

# dts package show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT
Transfer Package :
Label : PDQ0QMLNT
TransferSiteShippingAddress : Telehouse c/o Oracle Data Transfer Service; Christos Panoudis; Phone: +49152 22882673 Job:J5Y0R5KAS Package:PDQ0QMLNT ; Kleyerstraße 75-87; 60326 Frankfurt am Main; Germany
DeliveryVendor : DHL
DeliveryTrackingNumber : 5542918614
ReturnDeliveryTrackingNumber : 12345678
Status : PROCESSING
Devices :
[1] :
Label : DAMOED7GH
SerialNumber : WD-WX61E82FY633
UploadStatusLogURL : J5Y0R5KAS/DAMOED7GH/upload_summary.txt
Status : PROCESSING

# dts package show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT

Transfer Package :

Label : PDQ0QMLNT

DeliveryVendor : DHL

DeliveryTrackingNumber : 5542918614

ReturnDeliveryTrackingNumber : 12345678

Status : PROCESSING

Devices :

[1] :

Label : DAMOED7GH

SerialNumber : WD-WX61E82FY633

UploadStatusLogURL : J5Y0R5KAS/DAMOED7GH/upload_summary.txt

Status : PROCESSING

12. Data Verification

Finally the data is arrive in the Oracle Cloud Infrastructure Object Storage and is ready for use. The file processing is logged in the new created file upload_summary.txt.

# dts package show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT
Transfer Package :
Label : PDQ0QMLNT
TransferSiteShippingAddress : Telehouse c/o Oracle Data Transfer Service; Christos Panoudis; Phone: +49152 22882673 Job:J5Y0R5KAS Package:PDQ0QMLNT ; Kleyerstraße 75-87; 60326 Frankfurt am Main; Germany
DeliveryVendor : DHL
DeliveryTrackingNumber : 5542918614
ReturnDeliveryTrackingNumber : 12345678
Status : PROCESSING
Devices :
[1] :
Label : DAMOED7GH
SerialNumber : WD-WX61E82FY633
UploadStatusLogURL : J5Y0R5KAS/DAMOED7GH/upload_summary.txt
Status : PROCESSING


########################################################################################################################################################################################################
######################################### SUMMARY FOR DEVICE [WD-WX61E82FY633] ############################################
Generated at 2020-11-02 18:27:35
TOTAL: 8
### P present: 8
### M missing: 0
### C name collision: 0
### U unreadable: 0
### N nameTooLong: 0
########################################################################################################################################################################################################
| STATUS | NAME | LAST_MODIFIED | SIZE(MB) | MD5 | ETag |
| present | RMAN_CDB118_28102020.tgz | Mon Nov 02 18:27:36 UTC 2020 | 663.40 | 6ivSfbVKHceQ6FMy2xVSNA== | 99baa855-b15c-4f79-bfaa-dbf16385a182 |
| present | logFile.log | Mon Nov 02 18:27:16 UTC 2020 | 0.05 | Aepiez79Y7/WZYweOWA40w== | 894a88ee-c2f2-454b-875a-0bc3339ff6ab |
| present | bernmobil-opendata.tgz | Mon Nov 02 18:27:16 UTC 2020 | 0.13 | J6VvwZemr5ayF+hZmEMvUQ== | 38aa09fa-46f7-4009-8f8d-b710a43fe74f |
| present | expdphr-28-10-20-02.dmp.tgz | Mon Nov 02 18:27:29 UTC 2020 | 293.73 | tgtoAyh79EaAHnG2cMhJ4g== | 78094b66-6366-464a-9c93-3d4a4e1be3ca |
| present | basel-opendata.tgz | Mon Nov 02 18:27:17 UTC 2020 | 13.18 | rRKvfnfw3kjRgp+yyJd9eg== | f70fb2e9-baf2-40b1-a1dc-7522e6005d66 |
| present | bundesverwaltung-opendata.tgz | Mon Nov 02 18:27:19 UTC 2020 | 27.17 | Pbs8W5rNpqs+x48bb/wVQw== | 20f46883-2e13-4655-8c63-ff455db81fba |
| present | sbb-opendata.tgz | Mon Nov 02 18:27:26 UTC 2020 | 213.30 | iggsOeCuvpE+zDlwoYmXcA== | 90b81b13-611d-440a-9915-35c53e46b10b |
| present | expdphr-28-10-20-01.dmp.tgz | Mon Nov 02 18:27:29 UTC 2020 | 293.73 | tgtoAyh79EaAHnG2cMhJ4g== | 258c26c8-6d2f-4fba-864f-b03062de699e |
########################################################################################################################################################################################################

# dts package show --job-id ocid1.datatransferjob.oc1.eu-frankfurt-1.antheljszffzlryauwznondp2zp6rztpbvzyekv12345671234567 --package-label PDQ0QMLNT

Transfer Package :

Label : PDQ0QMLNT

DeliveryVendor : DHL

DeliveryTrackingNumber : 5542918614

ReturnDeliveryTrackingNumber : 12345678

Status : PROCESSING

Devices :

[1] :

Label : DAMOED7GH

SerialNumber : WD-WX61E82FY633

UploadStatusLogURL : J5Y0R5KAS/DAMOED7GH/upload_summary.txt

Status : PROCESSING

########################################################################################################################################################################################################

######################################### SUMMARY FOR DEVICE [WD-WX61E82FY633] ############################################

Generated at 2020-11-02 18:27:35

TOTAL: 8

### P present: 8

### M missing: 0

### C name collision: 0

### U unreadable: 0

### N nameTooLong: 0

13. Object Storage Replication Policy (optional)

The files are now in the data center Frankfurt, but I want to have them in the Swiss data center region Zurich. Therefore I set a replication policy on level Object Storage. In Zurich, a new bucket called data_transfer_usb_from_FRA is created. And a few minutes later, the files were available in the Object Storage Zurich. Sure, it depends on the file size 😉

Finally…

Detach the transfer disk so the data center guys can send it back to you.

And after a few days…welcome FedEx in Kestenholz / Jurasüdfuss / Solothurn / Switzerland!

Some words about Shipping and Costs

Shipping costs from DHL and Fedex:

Vendor	From	To	Costs
DHL	Langenthal / Switzerland	Frankfurt	79.50 CHF
FedEx	Frankfurt	Kestenholz	130.45 CHF

Summary

To watch nice marketing slides and documents about cool features is not enough. To find out how it works in the real word, a real is test is required. How to migrate data into a data center of any cloud provider should be basic know-how of each consultant which is working with and on cloud themes. Moving data by a disk ro an appliance opens a lot of possibilities for data migrations into the cloud. For example a huge DWH: Transfer the RMAN backup into the cloud, restore it, close the GAP by an incremental backup and synchronize it with Oracle Golden Gate. #ilike

Oracle Cloud Infrastructure

Oracle Cloud Infrastructure Classic Object Storage – Cleanup Day with FTM CLI

Author: Martin Berger 19. November 2020

Yesterday I decided to cleanup old Oracle Cloud Infrastructure Classic objects. There were a lot of files lying around in the Object Storage of a project from 2018. Cleaning up these files in the OCI console was no option, they can only be deleted one by one. And with over 1500 files, a bad idea. During the search for an option for Object Storage mass deletion, I found this tool: ftmcli – Object Storage Classic File Transfer Manager. The MOS note contains the script and a short manual how to use. It’s a Java based script, a perfect match for my Windows Subsytem for Linux (Ubuntu), which I often use for OCI actions.

OAC-Classic : How To Delete A Storage Container That has Multiple Objects. (Doc ID 2634021.1)

Link to the User Guide: https://docs.oracle.com/en/cloud/iaas-classic/storage-cloud/csclr/preparing-use-ftm-cli.html#GUID-5BB8647F-DDAD-4371-A519-1116402245FB

The Container List

Here is the content of my Object Storage Classic what I have to clean up – it contains five containers.

ftmcli – Installation

Over the WSL mountpoint in /mnt where you have access to your local Windows disk drives, transfer the package to the home directory and extract it.

mbg@DESKTOP-ID2UR6L:~$ unzip ftmcli-v2.4.3.zip
Archive:  ftmcli-v2.4.3.zip
   creating: ftmcli-v2.4.3/
  inflating: ftmcli-v2.4.3/README.txt
  inflating: ftmcli-v2.4.3/ftmcli.properties
  inflating: ftmcli-v2.4.3/ftmcli.jar

mbg@DESKTOP-ID2UR6L:~$ unzip ftmcli-v2.4.3.zip

Archive: ftmcli-v2.4.3.zip

creating: ftmcli-v2.4.3/

inflating: ftmcli-v2.4.3/README.txt

inflating: ftmcli-v2.4.3/ftmcli.properties

inflating: ftmcli-v2.4.3/ftmcli.jar

In the extracted subdirectory is a file called ftmcli.properties – there you have to set your OCI Object Storage information. Two parameters are used:

user=<OCI login>
rest-endpoint=<your Storage Classic Endpoint>

The Storage Classic Endpoint is visible in the Storage Classic Account tab. There is no need to set the password in the properties file. You are prompted for it when ftmcli is executed.

ftmcli – Commands

Available commands – output from ftmcli:

upload	Upload a file or a directory to a container.
download	Download an object or a virtual directory from a container.
create-container	Create a container.
restore	Restore an object from an Archive container.
list	List containers in the account or objects in a container.
delete	Delete a container in the account or an object in a container.
describe	Describes the attributes of a container in the account or an object in a container.
set	Set the metadata attribute(s) of a container in the account or an object in a container.
set-crp	Set a replication policy for a container.
copy	Copy an object to a destination container.

ftmcli – List Object Storage Containers

Here you will be prompted for your OCI password. Curious thing: My existing OCI password what I have used for months contained a lot of special characters( ,LCaOQ3|~[PT”+x), and there were not accepted.

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar list
Enter your password:
ERROR:Authentication failed. Authentication failure

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar list

Enter your password:

ERROR:Authentication failed. Authentication failure

I had to set a new OCI account password with less complexity and special characters and then it worked.

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar list
Enter your password:
_apaas
bdcvision
dbcs-vision
oac-vision
oracle-data-storagec-1

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar list

Enter your password:

_apaas

bdcvision

dbcs-vision

oac-vision

oracle-data-storagec-1

ftmcli – Delete Object Storage Containers

With the -f flag, the deletion of a container which contains objects can be forced. I did it for all my containers.

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar delete -f oracle-data-storagec-1
Enter your password:
Container successfully deleted: oracle-data-storagec-1

mbg@DESKTOP-ID2UR6L:~/ftmcli-v2.4.3$ java -jar ftmcli.jar delete -f oracle-data-storagec-1

Enter your password:

Container successfully deleted: oracle-data-storagec-1

And finally all containers are removed. Job successfully done!

Summary

This is what I like in Oracle Cloud Infrastructure, for a lot of problems and technical questions are a lot of scripts and tools available like this one: ftmcli. But sometimes it’s not so easy to find them.

Oracle ACE, Oracle Cloud Infrastructure, Oracle Tools, TrivadisContent

MV2OCI – One-Click Move of your Data into Oracle Cloud Infrastructure Database

Author: Martin Berger 30. October 2020

mv2oci is a tool which helps to migrate on-premise data to the Oracle Cloud Infrastructure based on Oracle Data Pump and works as a data load tool. The local Data Pump export is transferred and imported to/on the target cloud server automatically. There is no use of Oracle Cloud Object Storage, the dump files are transferred with rsync or scp to the target database node. This is the different behavior to mv2adb – see my blogpost here – which uses the Object Storage. As an option, the data can be transferred via Database Link (mv2oci Parameter –netlink).

All you need to know about mv2oci is written in the My Oracle Support Note (OCI) MV2OCI: move data to Oracle Cloud Database in “one-click” (Doc ID 2514026.1). The newest version of the rpm package can be downloaded there. The package has to be installed on the source server.

Prerequisites

SQL*Net connection between the two databases
A Java executable – in my case I have installed jre (yum install jre)
Verify if the firewall to the VCN Subnet is open for Port 1521 – Port 22 is open as per default
Password of database user SYSTEM

The Use Case

Let’s move the database schema SOE from my on-premise Oracle Linux Server into the cloud step-by-step. An Oracle Cloud Infrastructure database instance is already up and running, the target tablespace is created. The data centers are connect by VPN.

Database Information

	Source	Target
CDB Name	CDB118	CDB118
PDB Name	pdb11801	pdboci
Hostname	heckenweg	srv-cdb118
IP Address	192.168.1.184	172.16.0.8
PDB Service Name	pdb11801.kestenholz.net	pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com

1. Package Installation

Download and transfer the package to the on-premise server, for example in directory /tmp. As user root, install the package.

# rpm -ihv mv2oci-2.0.2-3.el7.x86_64.rpm
warning: mv2oci-2.0.2-3.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 939112d6: NOKEY
Verifying... ################################# [100%]
Preparing... ################################# [100%]
Updating / installing...
1:mv2oci-2.0.2-3 ################################# [100%]

mv2oci-2.0.2.3 binary has been installed on /opt/mv2oci succesfully!

# rpm -ihv mv2oci-2.0.2-3.el7.x86_64.rpm

warning: mv2oci-2.0.2-3.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 939112d6: NOKEY

Verifying... ################################# [100%]

Preparing... ################################# [100%]

Updating / installing...

1:mv2oci-2.0.2-3 ################################# [100%]

mv2oci-2.0.2.3 binary has been installed on /opt/mv2oci succesfully!

Verify that the SSH private key which is used for the connection to the Oracle Cloud Infrastructure server is available and the connection is working. Here is the OCI SSH key available in the $HOME/.ssh.

$ ssh -i $HOME/.ssh/id_rsa_oci_29012020 opc@172.16.0.8
Last login: Fri Oct 30 05:39:27 2020 from 192.168.1.235
[opc@srv-cdb118 ~]$

$ ssh -i $HOME/.ssh/id_rsa_oci_29012020 opc@172.16.0.8

Last login: Fri Oct 30 05:39:27 2020 from 192.168.1.235

[opc@srv-cdb118 ~]$

2. Encrypt the SYSTEM passwords for both databases – mv2oci encpass

# ./mv2oci.bin encpass

Please enter the password :
Please re-enter the password :
79F510449D6D43B028EB2E5FCC7CC71A336F363F123456789ABCDEFGHIJKLMNO

# ./mv2oci.bin encpass

Please enter the password :

Please re-enter the password :

79F510449D6D43B028EB2E5FCC7CC71A336F363F123456789ABCDEFGHIJKLMNO

3. Configuration File

A template of the configuration file is located in /opt/mv2oci/oci. I used the following parameters – other parameters like ICHOME for Instance Client configuration are well described.

Source DB Parameters

Parameter	Value
DB_CONSTRING	//heckenweg/pdb11801.kestenholz.net
SYSTEM_DB_PASSWORD	Encrypted SYSTEM password
SCHEMAS	SOE
DUMP_FILES	/tmp/exp_soe_18102020_01.dmp, /tmp/exp_soe_18102020_02.dmp
OHOME	/u01/app/oracle/product/19.0.0/dbhome_1

Expdp/Impdp Parameters

Parameter	Value
Dump Name	exp_soe_18102020.dmp
DUMP_PATH	/tmp
PARALLEL	2 – creates two Dumpfiles called exp_soe_18102020_01.dmp and exp_soe_18102020_02.dmp

OCI Parameters

Parameter	Value
OC_HOST	172.16.0.8
OC_SSHKEY	/home/oracle/.ssh/id_rsa_oci_29012020
OC_DB_CONSTRING	//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com
OC_DB_PASSWORD	Encrypted SYSTEM password
OC_DUMP_LOC	/tmp

4. Export Data – mv2oci expdp

# ./mv2oci.bin expdp --conf conf/CDB118.mv2oci.cfg

INFO: 2020-10-30 06:32:51: Please check the logfile '/opt/mv2oci/out/log/mv2oci_314519.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
mv2oci - Move data to Oracle Cloud Infrastructure (OCI)
Version: 2.0.2-3
Copyright (c) 1982-2019 Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton &lt;ruggero.citton@oracle.com&gt;
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:32:51: Reading configuration file 'conf/CDB118.mv2oci.cfg'
INFO: 2020-10-30 06:32:51: Using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'
INFO: 2020-10-30 06:32:51: Getting source DB version
INFO: 2020-10-30 06:32:52: Checking schemas on source database '//heckenweg/pdb11801.kestenholz.net'
INFO: 2020-10-30 06:32:53: Creating expdp directory 'MV2OCI_DP_DIR' for path '/tmp'
INFO: 2020-10-30 06:32:54: Getting latest SCN
INFO: 2020-10-30 06:32:55: Expdp parallelism set by parameter '2'
INFO: 2020-10-30 06:32:55: Executing Expdp as jobname 'MV2OCI_JOB'
INFO: 2020-10-30 06:32:55: Expdp log location available at the end of the process is '/opt/mv2oci/out/log/mv2oci_expdp_314519.log'

FLASHBACK automatically enabled to preserve database integrity.
Starting "SYSTEM"."MV2OCI_JOB": system/********@//heckenweg/pdb11801.kestenholz.net SILENT=BANNER JOB_NAME=MV2OCI_JOB SCHEMAS=SOE VERSION=19.0.0 DIRECTORY=MV2OCI_DP_DIR DUMPFILE=exp_soe_18102020_%u.dmp PARALLEL=2 REUSE_DUMPFILES=y LOGFILE=mv2oci_expdp_314519.log FILESIZE=4G FLASHBACK_SCN=6980091
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/TABLESPACE_QUOTA
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/STATISTICS/MARKER
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
. . exported "SOE"."ORDERS" 129.1 MB 1429790 rows
. . exported "SOE"."ORDER_ITEMS" 380.1 MB 7138088 rows
. . exported "SOE"."CUSTOMERS" 108.1 MB 1000000 rows
. . exported "SOE"."ADDRESSES" 110.0 MB 1500000 rows
. . exported "SOE"."CARD_DETAILS" 63.87 MB 1500000 rows
. . exported "SOE"."LOGON" 51.24 MB 2382984 rows
. . exported "SOE"."PRODUCT_DESCRIPTIONS" 220.8 KB 1000 rows
. . exported "SOE"."PRODUCT_INFORMATION" 188.6 KB 1000 rows
. . exported "SOE"."WAREHOUSES" 35.42 KB 1000 rows
. . exported "SOE"."ORDERENTRY_METADATA" 5.609 KB 4 rows
. . exported "SOE"."INVENTORIES" 15.27 MB 901957 rows
Master table "SYSTEM"."MV2OCI_JOB" successfully loaded/unloaded
******************************************************************************
Dump file set for SYSTEM.MV2OCI_JOB is:
/tmp/exp_soe_18102020_01.dmp
/tmp/exp_soe_18102020_02.dmp
Job "SYSTEM"."MV2OCI_JOB" successfully completed at 00:00

INFO: 2020-10-30 06:33:20: Following expdp dump has been created:
DUMP_FILES=/tmp/exp_soe_18102020_01.dmp,/tmp/exp_soe_18102020_02.dmp
SUCCESS: 2020-10-30 06:33:20: Expdp executed successfully

# ./mv2oci.bin expdp --conf conf/CDB118.mv2oci.cfg

INFO: 2020-10-30 06:32:51: Please check the logfile '/opt/mv2oci/out/log/mv2oci_314519.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

mv2oci - Move data to Oracle Cloud Infrastructure (OCI)

Version: 2.0.2-3

--------------------------------------------------------

Author: Ruggero Citton <ruggero.citton@oracle.com>

RAC Pack, Cloud Innovation and Solution Engineering Team

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:32:51: Reading configuration file 'conf/CDB118.mv2oci.cfg'

INFO: 2020-10-30 06:32:51: Using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'

INFO: 2020-10-30 06:32:51: Getting source DB version

INFO: 2020-10-30 06:32:52: Checking schemas on source database '//heckenweg/pdb11801.kestenholz.net'

INFO: 2020-10-30 06:32:53: Creating expdp directory 'MV2OCI_DP_DIR' for path '/tmp'

INFO: 2020-10-30 06:32:54: Getting latest SCN

INFO: 2020-10-30 06:32:55: Expdp parallelism set by parameter '2'

INFO: 2020-10-30 06:32:55: Executing Expdp as jobname 'MV2OCI_JOB'

INFO: 2020-10-30 06:32:55: Expdp log location available at the end of the process is '/opt/mv2oci/out/log/mv2oci_expdp_314519.log'

FLASHBACK automatically enabled to preserve database integrity.

Starting "SYSTEM"."MV2OCI_JOB": system/********@//heckenweg/pdb11801.kestenholz.net SILENT=BANNER JOB_NAME=MV2OCI_JOB SCHEMAS=SOE VERSION=19.0.0 DIRECTORY=MV2OCI_DP_DIR DUMPFILE=exp_soe_18102020_%u.dmp PARALLEL=2 REUSE_DUMPFILES=y LOGFILE=mv2oci_expdp_314519.log FILESIZE=4G FLASHBACK_SCN=6980091

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY

Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS

Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA

Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

Processing object type SCHEMA_EXPORT/USER

Processing object type SCHEMA_EXPORT/SYSTEM_GRANT

Processing object type SCHEMA_EXPORT/ROLE_GRANT

Processing object type SCHEMA_EXPORT/DEFAULT_ROLE

Processing object type SCHEMA_EXPORT/TABLESPACE_QUOTA

Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA

Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE

Processing object type SCHEMA_EXPORT/TABLE/TABLE

Processing object type SCHEMA_EXPORT/STATISTICS/MARKER

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX

Processing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX

Processing object type SCHEMA_EXPORT/VIEW/VIEW

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT

. . exported "SOE"."ORDERS" 129.1 MB 1429790 rows

. . exported "SOE"."ORDER_ITEMS" 380.1 MB 7138088 rows

. . exported "SOE"."CUSTOMERS" 108.1 MB 1000000 rows

. . exported "SOE"."ADDRESSES" 110.0 MB 1500000 rows

. . exported "SOE"."CARD_DETAILS" 63.87 MB 1500000 rows

. . exported "SOE"."LOGON" 51.24 MB 2382984 rows

. . exported "SOE"."PRODUCT_DESCRIPTIONS" 220.8 KB 1000 rows

. . exported "SOE"."PRODUCT_INFORMATION" 188.6 KB 1000 rows

. . exported "SOE"."WAREHOUSES" 35.42 KB 1000 rows

. . exported "SOE"."ORDERENTRY_METADATA" 5.609 KB 4 rows

. . exported "SOE"."INVENTORIES" 15.27 MB 901957 rows

Master table "SYSTEM"."MV2OCI_JOB" successfully loaded/unloaded

******************************************************************************

Dump file set for SYSTEM.MV2OCI_JOB is:

/tmp/exp_soe_18102020_01.dmp

/tmp/exp_soe_18102020_02.dmp

Job "SYSTEM"."MV2OCI_JOB" successfully completed at 00:00

INFO: 2020-10-30 06:33:20: Following expdp dump has been created:

DUMP_FILES=/tmp/exp_soe_18102020_01.dmp,/tmp/exp_soe_18102020_02.dmp

SUCCESS: 2020-10-30 06:33:20: Expdp executed successfully

Dump files created in /tmp.

# ls -la /tmp/*.dmp
-rw-r-----. 1 oracle oinstall 569135104 Oct 30 06:33 /tmp/exp_soe_18102020_01.dmp
-rw-r-----. 1 oracle oinstall 331845632 Oct 30 06:33 /tmp/exp_soe_18102020_02.dmp

# ls -la /tmp/*.dmp

-rw-r-----. 1 oracle oinstall 569135104 Oct 30 06:33 /tmp/exp_soe_18102020_01.dmp

-rw-r-----. 1 oracle oinstall 331845632 Oct 30 06:33 /tmp/exp_soe_18102020_02.dmp

5. Transfer Data – mv2oci putdump

# ./mv2oci.bin putdump --conf conf/CDB118.mv2oci.cfg
INFO: 2020-10-30 06:42:41: Please check the logfile '/opt/mv2oci/out/log/mv2oci_319424.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
mv2oci - Move data to Oracle Cloud Infrastructure (OCI)
Version: 2.0.2-3
Copyright (c) 1982-2019 Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton &lt;ruggero.citton@oracle.com&gt;
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:42:41: Reading configuration file 'conf/CDB118.mv2oci.cfg'
INFO: 2020-10-30 06:42:41: Checking OCHost '172.16.0.8' connectivity
INFO: 2020-10-30 06:42:42: Determining if using rsync or scp to transfer dump files
INFO: 2020-10-30 06:42:42: Using 'rsync' to transfer dump files
INFO: 2020-10-30 06:42:42: Making required dump dir '/tmp' on remote '172.16.0.8'
INFO: 2020-10-30 06:42:42: Uploading dump files over '172.16.0.8' using 'rsync'
INFO: 2020-10-30 06:42:42: Copying /tmp/exp_soe_18102020_01.dmp....
INFO: 2020-10-30 06:42:42: Copying /tmp/exp_soe_18102020_02.dmp....
SUCCESS: 2020-10-30 06:43:06: Dump file ' /tmp/exp_soe_18102020_02.dmp' uploaded successfully
SUCCESS: 2020-10-30 06:43:24: Dump file '/tmp/exp_soe_18102020_01.dmp' uploaded successfully
INFO: 2020-10-30 06:43:24: Setting required permission for dump dir '/tmp' on remote '172.16.0.8'
SUCCESS: 2020-10-30 06:43:28: Dump files uploaded over '172.16.0.8' successfully

# ./mv2oci.bin putdump --conf conf/CDB118.mv2oci.cfg

INFO: 2020-10-30 06:42:41: Please check the logfile '/opt/mv2oci/out/log/mv2oci_319424.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

mv2oci - Move data to Oracle Cloud Infrastructure (OCI)

Version: 2.0.2-3

--------------------------------------------------------

Author: Ruggero Citton <ruggero.citton@oracle.com>

RAC Pack, Cloud Innovation and Solution Engineering Team

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:42:41: Reading configuration file 'conf/CDB118.mv2oci.cfg'

INFO: 2020-10-30 06:42:41: Checking OCHost '172.16.0.8' connectivity

INFO: 2020-10-30 06:42:42: Determining if using rsync or scp to transfer dump files

INFO: 2020-10-30 06:42:42: Using 'rsync' to transfer dump files

INFO: 2020-10-30 06:42:42: Making required dump dir '/tmp' on remote '172.16.0.8'

INFO: 2020-10-30 06:42:42: Uploading dump files over '172.16.0.8' using 'rsync'

INFO: 2020-10-30 06:42:42: Copying /tmp/exp_soe_18102020_01.dmp....

INFO: 2020-10-30 06:42:42: Copying /tmp/exp_soe_18102020_02.dmp....

SUCCESS: 2020-10-30 06:43:06: Dump file ' /tmp/exp_soe_18102020_02.dmp' uploaded successfully

SUCCESS: 2020-10-30 06:43:24: Dump file '/tmp/exp_soe_18102020_01.dmp' uploaded successfully

INFO: 2020-10-30 06:43:24: Setting required permission for dump dir '/tmp' on remote '172.16.0.8'

SUCCESS: 2020-10-30 06:43:28: Dump files uploaded over '172.16.0.8' successfully

Files are available now on target server.

$ ls -la /tmp/*.dmp
-rw-rw-rw- 1 oracle oinstall 569135104 Oct 30 05:43 /tmp/exp_soe_18102020_01.dmp
-rw-rw-rw- 1 oracle oinstall 331845632 Oct 30 05:43 /tmp/exp_soe_18102020_02.dmp

$ ls -la /tmp/*.dmp

-rw-rw-rw- 1 oracle oinstall 569135104 Oct 30 05:43 /tmp/exp_soe_18102020_01.dmp

-rw-rw-rw- 1 oracle oinstall 331845632 Oct 30 05:43 /tmp/exp_soe_18102020_02.dmp

6. Import Data

Tablespace SOEDATA exists on target server, otherwise you can use to the EXTRA_IMPDP parameters in the mv2oci configuration file to do a remapping etc.

# ./mv2oci.bin impdp --conf conf/CDB118.mv2oci.cfg
INFO: 2020-10-30 06:49:33: Please check the logfile '/opt/mv2oci/out/log/mv2oci_322277.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
mv2oci - Move data to Oracle Cloud Infrastructure (OCI)
Version: 2.0.2-3
Copyright (c) 1982-2019 Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton &lt;ruggero.citton@oracle.com&gt;
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:49:33: Reading configuration file 'conf/CDB118.mv2oci.cfg'
INFO: 2020-10-30 06:49:33: Using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'
INFO: 2020-10-30 06:49:33: Expdp parallelism set by parameter '2'
INFO: 2020-10-30 06:49:33: Creating impdp directory 'MV2OCI_DP_DIR' for path '/tmp' on '172.16.0.8'
INFO: 2020-10-30 06:49:34: Executing import datapump as jobname 'MV2OCI_JOB'
INFO: 2020-10-30 06:49:34: Impdp log location available at the end of the process is '/opt/mv2oci/out/log/mv2oci_impdp_322277.log'
Master table "SYSTEM"."MV2OCI_JOB" successfully loaded/unloaded
Starting "SYSTEM"."MV2OCI_JOB": system/********@//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com SILENT=BANNER JOB_NAME=MV2OCI_JOB DIRECTORY=MV2OCI_DP_DIR DUMPFILE=exp_soe_18102020_01.dmp,exp_soe_18102020_02.dmp LOGFILE=mv2oci_impdp_322277.log PARALLEL=2
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/TABLESPACE_QUOTA
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
. . imported "SOE"."ORDERS" 129.1 MB 1429790 rows
. . imported "SOE"."CUSTOMERS" 108.1 MB 1000000 rows
. . imported "SOE"."ADDRESSES" 110.0 MB 1500000 rows
. . imported "SOE"."ORDER_ITEMS" 380.1 MB 7138088 rows
. . imported "SOE"."CARD_DETAILS" 63.87 MB 1500000 rows
. . imported "SOE"."LOGON" 51.24 MB 2382984 rows
. . imported "SOE"."PRODUCT_DESCRIPTIONS" 220.8 KB 1000 rows
. . imported "SOE"."PRODUCT_INFORMATION" 188.6 KB 1000 rows
. . imported "SOE"."WAREHOUSES" 35.42 KB 1000 rows
. . imported "SOE"."ORDERENTRY_METADATA" 5.609 KB 4 rows
. . imported "SOE"."INVENTORIES" 15.27 MB 901957 rows
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/STATISTICS/MARKER
ORA-39082: Object type PACKAGE BODY:"SOE"."ORDERENTRY" created with compilation warnings

Job "SYSTEM"."MV2OCI_JOB" completed with 1 error(s) at 00:00

ERROR: 2020-10-30 06:51:14: Executing impdp on remote '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

# ./mv2oci.bin impdp --conf conf/CDB118.mv2oci.cfg

INFO: 2020-10-30 06:49:33: Please check the logfile '/opt/mv2oci/out/log/mv2oci_322277.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

mv2oci - Move data to Oracle Cloud Infrastructure (OCI)

Version: 2.0.2-3

--------------------------------------------------------

Author: Ruggero Citton <ruggero.citton@oracle.com>

RAC Pack, Cloud Innovation and Solution Engineering Team

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 06:49:33: Reading configuration file 'conf/CDB118.mv2oci.cfg'

INFO: 2020-10-30 06:49:33: Using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'

INFO: 2020-10-30 06:49:33: Expdp parallelism set by parameter '2'

INFO: 2020-10-30 06:49:33: Creating impdp directory 'MV2OCI_DP_DIR' for path '/tmp' on '172.16.0.8'

INFO: 2020-10-30 06:49:34: Executing import datapump as jobname 'MV2OCI_JOB'

INFO: 2020-10-30 06:49:34: Impdp log location available at the end of the process is '/opt/mv2oci/out/log/mv2oci_impdp_322277.log'

Master table "SYSTEM"."MV2OCI_JOB" successfully loaded/unloaded

Starting "SYSTEM"."MV2OCI_JOB": system/********@//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com SILENT=BANNER JOB_NAME=MV2OCI_JOB DIRECTORY=MV2OCI_DP_DIR DUMPFILE=exp_soe_18102020_01.dmp,exp_soe_18102020_02.dmp LOGFILE=mv2oci_impdp_322277.log PARALLEL=2

Processing object type SCHEMA_EXPORT/USER

Processing object type SCHEMA_EXPORT/SYSTEM_GRANT

Processing object type SCHEMA_EXPORT/ROLE_GRANT

Processing object type SCHEMA_EXPORT/DEFAULT_ROLE

Processing object type SCHEMA_EXPORT/TABLESPACE_QUOTA

Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA

Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE

Processing object type SCHEMA_EXPORT/TABLE/TABLE

Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA

. . imported "SOE"."ORDERS" 129.1 MB 1429790 rows

. . imported "SOE"."CUSTOMERS" 108.1 MB 1000000 rows

. . imported "SOE"."ADDRESSES" 110.0 MB 1500000 rows

. . imported "SOE"."ORDER_ITEMS" 380.1 MB 7138088 rows

. . imported "SOE"."CARD_DETAILS" 63.87 MB 1500000 rows

. . imported "SOE"."LOGON" 51.24 MB 2382984 rows

. . imported "SOE"."PRODUCT_DESCRIPTIONS" 220.8 KB 1000 rows

. . imported "SOE"."PRODUCT_INFORMATION" 188.6 KB 1000 rows

. . imported "SOE"."WAREHOUSES" 35.42 KB 1000 rows

. . imported "SOE"."ORDERENTRY_METADATA" 5.609 KB 4 rows

. . imported "SOE"."INVENTORIES" 15.27 MB 901957 rows

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC

Processing object type SCHEMA_EXPORT/VIEW/VIEW

Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY

Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX

Processing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT

Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS

Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS

Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT

Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

Processing object type SCHEMA_EXPORT/STATISTICS/MARKER

ORA-39082: Object type PACKAGE BODY:"SOE"."ORDERENTRY" created with compilation warnings

Job "SYSTEM"."MV2OCI_JOB" completed with 1 error(s) at 00:00

ERROR: 2020-10-30 06:51:14: Executing impdp on remote '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

Analysis of the error in the SQL*Developer – there is a missing execution permission on package DBMS_LOCK.

This is an easy thing:

SQL&gt; GRANT EXECUTE ON dbms_lock TO SOE;

Grant succeeded.

SQL> GRANT EXECUTE ON dbms_lock TO SOE;

Grant succeeded.

7. Reporting – mv2oci report

The report compares the objects on source and target database.

# ./mv2oci.bin report --conf conf/CDB118.mv2oci.cfg
INFO: 2020-10-30 07:00:13: Please check the logfile '/opt/mv2oci/out/log/mv2oci_327866.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
mv2oci - Move data to Oracle Cloud Infrastructure (OCI)
Version: 2.0.2-3
Copyright (c) 1982-2019 Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton &lt;ruggero.citton@oracle.com&gt;
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 07:00:13: Reading configuration file 'conf/CDB118.mv2oci.cfg'
INFO: 2020-10-30 07:00:14: using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'
INFO: 2020-10-30 07:00:14: Getting Obj report for '//heckenweg/pdb11801.kestenholz.net'

0
Database Objects REPORT for //heckenweg/pdb11801.kestenholz.net

 SOE       |INDEX       |        27
 SOE       |PACKAGE     |         1
 SOE       |TABLE       |        11
 SOE       |VIEW        |         2
 SOE       |SEQUENCE    |         5
 SOE       |PACKAGE BODY|         1
           |            |----------
 sum       |            |        47

INFO: 2020-10-30 07:00:15: Getting Segments report for '//heckenweg/pdb11801.kestenholz.net'

0
         Database Segments REPORT for //heckenweg/pdb11801.kestenholz.net

 SOE       |PRD_DESC_PK                   |INDEX          |      .125
 SOE       |WHS_LOCATION_IX               |INDEX          |     .0625
 SOE       |ORD_CUSTOMER_IX               |INDEX          |   27.1875
 SOE       |INVENTORIES                   |TABLE          |       176
 SOE       |ORD_WAREHOUSE_IX              |INDEX          |   30.1875
 SOE       |INVENTORY_PK                  |INDEX          |        18
 SOE       |CARD_DETAILS                  |TABLE          |        80
 SOE       |ORDERS                        |TABLE          |       160
 SOE       |CUSTOMERS_PK                  |INDEX          |    18.625
 SOE       |PROD_SUPPLIER_IX              |INDEX          |      .125
 SOE       |PRODUCT_INFORMATION           |TABLE          |       .25
 SOE       |ORDER_ITEMS_PK                |INDEX          |     146.5
 SOE       |ORDER_PK                      |INDEX          |   25.9375
 SOE       |ORD_ORDER_DATE_IX             |INDEX          |    31.875
 SOE       |INV_WAREHOUSE_IX              |INDEX          |   16.6875
 SOE       |ORDERENTRY_METADATA           |TABLE          |     .0625
 SOE       |CARD_DETAILS_PK               |INDEX          |   27.1875
 SOE       |ORD_SALES_REP_IX              |INDEX          |   25.5625
 SOE       |PROD_CATEGORY_IX              |INDEX          |     .0625
 SOE       |CUST_ACCOUNT_MANAGER_IX       |INDEX          |    18.625
 SOE       |CUSTOMERS                     |TABLE          |       136
 SOE       |ADDRESSES                     |TABLE          |       136
 SOE       |ORDER_ITEMS                   |TABLE          |       464
 SOE       |PRODUCT_INFORMATION_PK        |INDEX          |     .0625
 SOE       |ADDRESS_CUST_IX               |INDEX          |   28.4375
 SOE       |ITEM_PRODUCT_IX               |INDEX          |  120.5625
 SOE       |LOGON                         |TABLE          |        64
 SOE       |PRODUCT_DESCRIPTIONS          |TABLE          |     .3125
 SOE       |ADDRESS_PK                    |INDEX          |    27.125
 SOE       |WAREHOUSES_PK                 |INDEX          |     .0625
 SOE       |PROD_NAME_IX                  |INDEX          |      .125
 SOE       |CARDDETAILS_CUST_IX           |INDEX          |   28.4375
 SOE       |WAREHOUSES                    |TABLE          |     .0625
 SOE       |ITEM_ORDER_IX                 |INDEX          |   130.875
 SOE       |INV_PRODUCT_IX                |INDEX          |     16.75
 SOE       |CUST_DOB_IX                   |INDEX          |   23.1875
 SOE       |CUST_EMAIL_IX                 |INDEX          |     41.25
 SOE       |CUST_FUNC_LOWER_NAME_IX       |INDEX          |   29.0625
           |                              |               |----------
 sum       |                              |               |  2049.375

INFO: 2020-10-30 07:00:16: Getting Obj report for '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

0
 Database Objects REPORT for //172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.or

 SOE       |INDEX       |        27
 SOE       |PACKAGE     |         1
 SOE       |SEQUENCE    |         5
 SOE       |TABLE       |        11
 SOE       |VIEW        |         2
 SOE       |PACKAGE BODY|         1
           |            |----------
 sum       |            |        47


INFO: 2020-10-30 07:00:17: Getting Segments report for '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

0
 Database Segments REPORT for //172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.o

 SOE       |WHS_LOCATION_IX               |INDEX          |     .0625
 SOE       |PRD_DESC_PK                   |INDEX          |     .0625
 SOE       |ORD_CUSTOMER_IX               |INDEX          |        26
 SOE       |INVENTORIES                   |TABLE          |       176
 SOE       |ORD_WAREHOUSE_IX              |INDEX          |        29
 SOE       |INVENTORY_PK                  |INDEX          |        18
 SOE       |ORDERS                        |TABLE          |       160
 SOE       |CARD_DETAILS                  |TABLE          |        80
 SOE       |PROD_SUPPLIER_IX              |INDEX          |     .0625
 SOE       |CUSTOMERS_PK                  |INDEX          |        17
 SOE       |PRODUCT_INFORMATION           |TABLE          |       .25
 SOE       |ORDER_ITEMS_PK                |INDEX          |       144
 SOE       |ORDER_PK                      |INDEX          |        25
 SOE       |ORD_ORDER_DATE_IX             |INDEX          |        31
 SOE       |INV_WAREHOUSE_IX              |INDEX          |        15
 SOE       |ORDERENTRY_METADATA           |TABLE          |     .0625
 SOE       |PROD_CATEGORY_IX              |INDEX          |     .0625
 SOE       |ORD_SALES_REP_IX              |INDEX          |        24
 SOE       |CUST_ACCOUNT_MANAGER_IX       |INDEX          |        17
 SOE       |CARD_DETAILS_PK               |INDEX          |        26
 SOE       |ADDRESSES                     |TABLE          |       136
 SOE       |CUSTOMERS                     |TABLE          |       136
 SOE       |ORDER_ITEMS                   |TABLE          |       464
 SOE       |ADDRESS_CUST_IX               |INDEX          |        27
 SOE       |ITEM_PRODUCT_IX               |INDEX          |       120
 SOE       |PRODUCT_INFORMATION_PK        |INDEX          |     .0625
 SOE       |PRODUCT_DESCRIPTIONS          |TABLE          |     .3125
 SOE       |LOGON                         |TABLE          |        64
 SOE       |WAREHOUSES_PK                 |INDEX          |     .0625
 SOE       |ADDRESS_PK                    |INDEX          |        26
 SOE       |PROD_NAME_IX                  |INDEX          |      .125
 SOE       |CARDDETAILS_CUST_IX           |INDEX          |        27
 SOE       |WAREHOUSES                    |TABLE          |     .0625
 SOE       |ITEM_ORDER_IX                 |INDEX          |       136
 SOE       |CUST_DOB_IX                   |INDEX          |        22
 SOE       |CUST_EMAIL_IX                 |INDEX          |        40
 SOE       |INV_PRODUCT_IX                |INDEX          |        15
 SOE       |CUST_FUNC_LOWER_NAME_IX       |INDEX          |        28
           |                              |               |----------
 sum       |                              |               | 2030.1875

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

# ./mv2oci.bin report --conf conf/CDB118.mv2oci.cfg

INFO: 2020-10-30 07:00:13: Please check the logfile '/opt/mv2oci/out/log/mv2oci_327866.log' for more details

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

mv2oci - Move data to Oracle Cloud Infrastructure (OCI)

Version: 2.0.2-3

--------------------------------------------------------

Author: Ruggero Citton <ruggero.citton@oracle.com>

RAC Pack, Cloud Innovation and Solution Engineering Team

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2020-10-30 07:00:13: Reading configuration file 'conf/CDB118.mv2oci.cfg'

INFO: 2020-10-30 07:00:14: using Oracle Home '/u01/app/oracle/product/19.0.0/dbhome_1/'

INFO: 2020-10-30 07:00:14: Getting Obj report for '//heckenweg/pdb11801.kestenholz.net'

Database Objects REPORT for //heckenweg/pdb11801.kestenholz.net

SOE |INDEX | 27

SOE |PACKAGE | 1

SOE |TABLE | 11

SOE |VIEW | 2

SOE |SEQUENCE | 5

SOE |PACKAGE BODY| 1

| |----------

sum | | 47

INFO: 2020-10-30 07:00:15: Getting Segments report for '//heckenweg/pdb11801.kestenholz.net'

Database Segments REPORT for //heckenweg/pdb11801.kestenholz.net

SOE |PRD_DESC_PK |INDEX | .125

SOE |WHS_LOCATION_IX |INDEX | .0625

SOE |ORD_CUSTOMER_IX |INDEX | 27.1875

SOE |INVENTORIES |TABLE | 176

SOE |ORD_WAREHOUSE_IX |INDEX | 30.1875

SOE |INVENTORY_PK |INDEX | 18

SOE |CARD_DETAILS |TABLE | 80

SOE |ORDERS |TABLE | 160

SOE |CUSTOMERS_PK |INDEX | 18.625

SOE |PROD_SUPPLIER_IX |INDEX | .125

SOE |PRODUCT_INFORMATION |TABLE | .25

SOE |ORDER_ITEMS_PK |INDEX | 146.5

SOE |ORDER_PK |INDEX | 25.9375

SOE |ORD_ORDER_DATE_IX |INDEX | 31.875

SOE |INV_WAREHOUSE_IX |INDEX | 16.6875

SOE |ORDERENTRY_METADATA |TABLE | .0625

SOE |CARD_DETAILS_PK |INDEX | 27.1875

SOE |ORD_SALES_REP_IX |INDEX | 25.5625

SOE |PROD_CATEGORY_IX |INDEX | .0625

SOE |CUST_ACCOUNT_MANAGER_IX |INDEX | 18.625

SOE |CUSTOMERS |TABLE | 136

SOE |ADDRESSES |TABLE | 136

SOE |ORDER_ITEMS |TABLE | 464

SOE |PRODUCT_INFORMATION_PK |INDEX | .0625

SOE |ADDRESS_CUST_IX |INDEX | 28.4375

SOE |ITEM_PRODUCT_IX |INDEX | 120.5625

SOE |LOGON |TABLE | 64

SOE |PRODUCT_DESCRIPTIONS |TABLE | .3125

SOE |ADDRESS_PK |INDEX | 27.125

SOE |WAREHOUSES_PK |INDEX | .0625

SOE |PROD_NAME_IX |INDEX | .125

SOE |CARDDETAILS_CUST_IX |INDEX | 28.4375

SOE |WAREHOUSES |TABLE | .0625

SOE |ITEM_ORDER_IX |INDEX | 130.875

SOE |INV_PRODUCT_IX |INDEX | 16.75

SOE |CUST_DOB_IX |INDEX | 23.1875

SOE |CUST_EMAIL_IX |INDEX | 41.25

SOE |CUST_FUNC_LOWER_NAME_IX |INDEX | 29.0625

| | |----------

sum | | | 2049.375

INFO: 2020-10-30 07:00:16: Getting Obj report for '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

Database Objects REPORT for //172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.or

SOE |INDEX | 27

SOE |PACKAGE | 1

SOE |SEQUENCE | 5

SOE |TABLE | 11

SOE |VIEW | 2

SOE |PACKAGE BODY| 1

| |----------

sum | | 47

INFO: 2020-10-30 07:00:17: Getting Segments report for '//172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.oraclevcn.com'

Database Segments REPORT for //172.16.0.8/pdboci.subnetvcnmohnwe.vcnmohnwegvpn.o

SOE |WHS_LOCATION_IX |INDEX | .0625

SOE |PRD_DESC_PK |INDEX | .0625

SOE |ORD_CUSTOMER_IX |INDEX | 26

SOE |INVENTORIES |TABLE | 176

SOE |ORD_WAREHOUSE_IX |INDEX | 29

SOE |INVENTORY_PK |INDEX | 18

SOE |ORDERS |TABLE | 160

SOE |CARD_DETAILS |TABLE | 80

SOE |PROD_SUPPLIER_IX |INDEX | .0625

SOE |CUSTOMERS_PK |INDEX | 17

SOE |PRODUCT_INFORMATION |TABLE | .25

SOE |ORDER_ITEMS_PK |INDEX | 144

SOE |ORDER_PK |INDEX | 25

SOE |ORD_ORDER_DATE_IX |INDEX | 31

SOE |INV_WAREHOUSE_IX |INDEX | 15

SOE |ORDERENTRY_METADATA |TABLE | .0625

SOE |PROD_CATEGORY_IX |INDEX | .0625

SOE |ORD_SALES_REP_IX |INDEX | 24

SOE |CUST_ACCOUNT_MANAGER_IX |INDEX | 17

SOE |CARD_DETAILS_PK |INDEX | 26

SOE |ADDRESSES |TABLE | 136

SOE |CUSTOMERS |TABLE | 136

SOE |ORDER_ITEMS |TABLE | 464

SOE |ADDRESS_CUST_IX |INDEX | 27

SOE |ITEM_PRODUCT_IX |INDEX | 120

SOE |PRODUCT_INFORMATION_PK |INDEX | .0625

SOE |PRODUCT_DESCRIPTIONS |TABLE | .3125

SOE |LOGON |TABLE | 64

SOE |WAREHOUSES_PK |INDEX | .0625

SOE |ADDRESS_PK |INDEX | 26

SOE |PROD_NAME_IX |INDEX | .125

SOE |CARDDETAILS_CUST_IX |INDEX | 27

SOE |WAREHOUSES |TABLE | .0625

SOE |ITEM_ORDER_IX |INDEX | 136

SOE |CUST_DOB_IX |INDEX | 22

SOE |CUST_EMAIL_IX |INDEX | 40

SOE |INV_PRODUCT_IX |INDEX | 15

SOE |CUST_FUNC_LOWER_NAME_IX |INDEX | 28

| | |----------

sum | | | 2030.1875

8. All in One – mvoci auto

We did the steps one-by-one, by using the parameter auto, the steps above are done automatically (except reporting).

# ./mv2oci.bin auto --conf conf/CDB118.mv2oci.cfg

1	# ./mv2oci.bin auto --conf conf/CDB118.mv2oci.cfg

9. Logfiles

Logfiles tom the mv2oci actions are located in:

mv2oci	/opt/mv2oci/out/log
Data Pump	Directory in parameter DUMP_PATH

Summary

mv2oci is another great tool to support the movement to Oracle Cloud Infrastructure. Easy to configure, easy to use. #ilike

Oracle ACE, Oracle Cloud Infrastructure, Oracle Linux, Oracle RDBMS, TrivadisContent

Oracle Cloud Infrastructure

Oracle Cloud Infrastructure – A short Blog Post about a secure and small Development Setup

Requirements

Architecture Diagram

Network Components

Monitoring

Links

Next Steps

Other Ideas

Summary

Oracle Cloud Infrastructure Data Safe – How to burn down 201.44 Swiss Francs in 30 Seconds…

Is Data Safe really for free?

Oracle Cloud Infrastructure Price List

Cost and Usage Report

Lessons learned

Oracle OCI Data Transfer Service – A journey from Kestenholz/Jurasüdfuss/Switzerland to Frankfurt and back

Setup

Data Transfer Service Regions

How is your data coming into the Oracle Cloud

1. Enable Data Transfer Service – Entitlement

2. Prepare an Object Storage Bucket

3. Create a Transfer Job

4. Attach a HDD to a Linux based host, use the Data Transfer Utility to create and encrypt the Device

Package Installation for my Oracle Linux 7 Machine

Download and Installation of the Data Transfer Utility

Configure IAM Credentials for Data Transfer Actions

Verify Credentials

Show Data Transfer Job Details – Status is PREPARED

Prepare USB Hard Disk Drive

Create Transfer Disk for Data Copy

5. Copy Data to HDD

6. Generate Manifest File

7. Lock the Disk

8. Attach the Disk to the Transfer Label and create the Package

9. Shipping and Shipping Information Update

10. Tracking

11. Data Processing

12. Data Verification

13. Object Storage Replication Policy (optional)

Finally…

Some words about Shipping and Costs

Links

Summary

Oracle Cloud Infrastructure Classic Object Storage – Cleanup Day with FTM CLI

The Container List

ftmcli – Installation

ftmcli – Commands

ftmcli – List Object Storage Containers

ftmcli – Delete Object Storage Containers

Summary

MV2OCI – One-Click Move of your Data into Oracle Cloud Infrastructure Database

Prerequisites

The Use Case

Database Information

1. Package Installation

2. Encrypt the SYSTEM passwords for both databases – mv2oci encpass

3. Configuration File

Source DB Parameters

Expdp/Impdp Parameters

OCI Parameters

4. Export Data – mv2oci expdp

5. Transfer Data – mv2oci putdump

6. Import Data

7. Reporting – mv2oci report

8. All in One – mvoci auto

9. Logfiles

Summary