Howto backup Oracle Cloud Infrastructure metadata

This post is also available in: Português

As of today (October 2018), there is still no internal capability in Oracle Cloud Infrastructure to backup the metadata of all the objects you have (like computes, volumes, network, users, etc). This would be indeed a great feature and prefer to believe that Oracle is working to deploy it.

Meanwhile, I have some customers that work with OCI and have the requisite to take backups of all cloud metadata information in case someone accidentally delete an important, like a security list rule, policy privilege or a compute tag, just to mention a few.

As the steps to manually backup all the objects metadata are not trivial, I've developed a script that will do all the work.

The script is public available at my github oci-scripts page: oci_json_export.sh

How does it work?

The “oci_json_export.sh” is a public tool that extracts all the Oracle Cloud Infrastructure information into json files and compact in a single zip output.

• It uses oci-cli and jq in the backend.
• Each execution of oci_json_export.sh may take several minutes, however, it will not impact the OCI performance.
• Execution time is long since oci_json_export.sh makes use of more than a hundred single-threaded sequentially-executed oci-cli calls to produce its output.
• For more information about oci_json_export.sh, refer to content and links at https://github.com/dbarj/oci-scripts/blob/master/oci_json_export.sh

Pre-requisites

As already said, oci_json_export.sh depends that you have a bash shell, oci-cli and jq.

[user@localhost sh]$ bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"

[user@localhost ~]$ cd ~/bin
[user@localhost ~]$ wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64
100%[===========================================>] 2.89M 862KB/s in 3.8s
2018-09-17 18:55:00 (776 KB/s) - ‘jq’ saved [3027945/3027945]
[user@localhost ~]$ chmod +x jq
[user@localhost ~]$ export PATH=$PATH:`pwd`
[user@localhost ~]$ which jq
/Users/myuser/bin/jq

How to run it?

1 - Download the latest version of the public tool oci_json_export.sh from https://raw.githubusercontent.com/dbarj/oci- scripts/master/oci_json_export.sh

2 - Copy to server (into any directory) where it will be executed. Optionally, you
can download it directly to the server using wget:

[user@localhost ~]$ wget https://raw.githubusercontent.com/dbarj/oci-scripts/master/oci_json_export.sh
100%[============================>]  26.39K  --.-KB/s    in 0.03s   
2018-10-10 11:00:21 (979 KB/s) - ‘oci_json_export.sh’ saved [27027/27027]
[user@localhost ~]$ chmod +x oci_json_export.sh

3 - Execute shell script oci_json_export.sh:

[user@localhost ~]$ ./oci_json_export.sh 
Usage: ./oci_json_export.sh <option>

<option> - Execution Scope.

Valid <option> values are:
- ALL         - Execute json export for ALL possible options and compress output in a zip file.
- ALL_REGIONS - Same as ALL, but run for all tenancy's subscribed regions.
- Audit-Events
- BV-BVBackups
- BV-BVolumes
...

4 - Note you need a parameter. The 1^st parameter must be either:

• ALL - Means it will scan and export all objects found in the current configured default region. Then it will zip the result in a single file.
• ALL_REGIONS - Means the same as ALL, but will go deeper and generate a ZIP for each region you are subscribed.
• (Specific Value) - There are more than 30+ other individually specific options you can pass and, in this case, it will export only the json for that object. The exported objected will be spooled to the stdout.

Execution Example

Below is a execution demo of the tool:

[user@localhost ~]$ ls -l oci_json_export.sh
-rwxrwx---. 1 user user 25192 Sep 17 15:28 oci_json_export.sh
[user@localhost ~]$ ./oci_json_export.sh ALL_REGIONS
Region us-ashburn-1 set.
Processing "oci_bv_boot-volume-backup.json".
Processing "oci_bv_boot-volume.json".
Processing "oci_bv_backup.json".
Processing "oci_bv_volume-backup-policy.json".
Processing "oci_bv_volume-backup-policy-assignment.json".
Processing "oci_bv_volume-group.json".
Processing "oci_bv_volume-group-backup.json".
Processing "oci_bv_volume.json".
Processing "oci_compute_boot-volume-attachment.json".
...

Optionally, you can export the variable DEBUG=1 before calling oci_json_export.sh script. You will be able to view execution steps:

[user@localhost ~]$ export DEBUG=1
[user@localhost ~]$ ./oci_json_export.sh ALL_REGIONS
Region us-ashburn-1 set.
Processing "oci_bv_boot-volume-backup.json".
Processing "oci_bv_boot-volume.json".
Processing "oci_bv_backup.json".
...

## (In another Shell)

[user@localhost ~]$ tail -f debug.log
20181010110704: oci --cli-rc-file /dev/null iam region-subscription list
20181010110706: oci --cli-rc-file /dev/null --region us-ashburn-1 iam compartment list --all
20181010110707: oci --cli-rc-file /dev/null --region us-ashburn-1 bv boot-volume-backup list --all --compartment-id ocid1.compartment.oc1..aaa
20181010110709: oci --cli-rc-file /dev/null --region us-ashburn-1 bv boot-volume-backup list --all --compartment-id ocid1.compartment.oc1..bbb
20181010110721: oci --cli-rc-file /dev/null --region us-ashburn-1 bv boot-volume-backup list --all --compartment-id ocid1.compartment.oc1..ccc
20181010110724: oci --cli-rc-file /dev/null --region us-ashburn-1 bv boot-volume-backup list --all --compartment-id ocid1.compartment.oc1..ddd
20181010110725: oci --cli-rc-file /dev/null --region us-ashburn-1 iam availability-domain list
...

Hope you enjoyed.

+8