Skip to content

Commit db02137

Browse files
oheimburgeroheimburger
authored
Merge pull request #30 from oracle-devrel/block.storage.01.00.28.06.2023
initial block storage version 1.0
2 parents fef1635 + 4983b29 commit db02137

File tree

7 files changed

+819
-0
lines changed

7 files changed

+819
-0
lines changed

‎cloud-infrastructure/storage/block-storage/README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Block Storage
2+
3+
High-performance block storage at any scale.
4+
5+
We've designed our storage platform as an ideal complement to Oracle compute and networking services to support the highest performance requirements. OCI Block Volume uses the latest NVMe SSDs and provides nonblocking network connectivity to every host. Oracle delivers a consistent, low-latency performance of up to 225 IOPS/GB to a maximum of 300,000 IOPS and 2,680 MB/sec of throughput per volume.
6+
7+
8+
# Table of Contents
9+
10+
1. [Team Publications](#team-publications)
11+
2. [Useful Links](#useful-uinks)
12+
3. [Reusable Assets Overview](#reusable-assets-overview)
13+
14+
## Team Publications
15+
16+
- [OCI provides native high availability and data resilience](https://blogs.oracle.com/cloud-infrastructure/post/oci-provide-cloud-resilience-by-default)
17+
- Blog: Oracle Cloud Infrastructure (OCI) provides high data durability and availability as a core feature, enabling businesses to concentrate on their customer experience and economic success.
18+
- [Make your cloud resilient against regional outages](https://www.youtube.com/watch?v=IVqLe_XH_AE)
19+
- 6-minute Video about "Make your cloud resilient against regional outages".
20+
- [Effortless Cloud Resilience – Automate your Disaster Recovery with Oracle Cloud Infrastructure](https://www.youtube.com/watch?v=P3qWyjE9HMQ)
21+
- 1h Video about "Effortless Cloud Resilience – Automate your Disaster Recovery with Oracle Cloud Infrastructure".
22+
23+
## Useful Links
24+
25+
- [OCI Block Storage documentation](https://docs.oracle.com/en-us/iaas/Content/Block/home.htm)
26+
- Oracle Block Storage documentation.
27+
- Introduction to Oracle Cloud Infrastructure Block Volume [Part 01](https://www.youtube.com/watch?v=rNrBxdDC8vc) and [Part 02](https://www.youtube.com/watch?v=ldZDySWv8sw)
28+
- Short Videos to introduce Oracle Cloud Infrastructure Block Volume.
29+
- [OCI Block Storage on oracle.com](https://www.oracle.com/cloud/storage/block-volumes/)
30+
- Oracle Cloud Infrastructure (OCI) Block Volumes provide reliable, high-performance, low-cost block storage. OCI Block Volumes persist beyond the lifespan of a virtual machine, have built-in redundancy, and can scale to 1 PB per compute instance.
31+
- [Blogs on Block Storage](https://blogs.oracle.com/authors/max-verun)
32+
- See all Block Storage Blogs from Oracle's Product Management.
33+
- [Block Storage Release Notes](https://docs.oracle.com/en-us/iaas/releasenotes/services/blockvolume/)
34+
- [Block Volumes FAQ](https://www.oracle.com/cloud/storage/block-volumes/faq)
35+
- [Oracle Cloud Infrastructure Service Level Agreement (SLA)](https://www.oracle.com/cloud/sla/)
36+
- Enterprises demand more than just availability from their cloud infrastructure. Mission-critical workloads also require consistent performance, and the ability to manage, monitor, and modify resources running in the cloud at any time. Only Oracle offers end-to-end SLAs covering the performance, availability, and manageability of services.
37+
- [Oracle PaaS and IaaS Public Cloud Services Pillar Document Block Storage](https://www.oracle.com/assets/paas-iaas-pub-cld-srvs-pillar-4021422.pdf#page=28)
38+
- Oracle offers end-to-end SLAs covering performance, availability, and manageability. Here you find details regarding Block Storage.
39+
- [Oracle PaaS and IaaS Universal Credits Service Descriptions - Storage section](https://www.oracle.com/us/corporate/contracts/paas-iaas-universal-credits-3940775.pdf#page=178)
40+
- OCI Block Storage Service Description.
41+
- [Resilience and Continuous Availability of Oracle Cloud Infrastructure Services and Platform FAQ](https://www.oracle.com/cloud/iaas/faq.html)
42+
- Resilience and Continuous Availability FAQs.
43+
- [Oracle Cloud Infrastructure Vault: Block Volume Encryption](https://www.youtube.com/watch?v=3GBPIx4hlRU)
44+
- This video helps you to encrypt a block volume with the KMS Vault customer-managed encryption key.
45+
46+
## Reusable Assets Overview
47+
48+
- [Cloud Resilience](https://gitlab.com/hmielimo/cloud-resilience/-/blob/main/doc/cloud.resilience/README.md)
49+
- All details regarding storage-based resiliency incl. example backup and recovery automation script.
50+
- [Cloud Resilience by default](https://gitlab.com/hmielimo/cloud-resilience-by-default/)
51+
- All details regarding storage-based Server resiliency including
52+
- Set up your Server with Resilience by default Using the Console
53+
- Set up your Server with Resilience by default using CLI
54+
- Performance-Based Auto-tune Use Case script
55+
- [OCI Storage Health-Check](https://gitlab.com/hmielimo/oci-storage-health-check/)
56+
- Storage Health-Check example scripts
57+
- validate all boot and block volumes regarding backup policies
58+
- showcase boot and block volume security e.g. using Customer managed key, Key-Rotation, Backup
59+
- [TRIM showcase - example script](asset/trim-showcase.md)
60+
- [boot/block volume security best practice - example script](asset/secure.storage.sh)
61+
- [boot/block volume individual (to a customer-managed bucket) backup and restore - example script](asset/block-volume-backup-and-restore.md)
62+
63+
64+
65+
# License
66+
67+
Copyright (c) 2023 Oracle and/or its affiliates.
68+
69+
Licensed under the Universal Permissive License (UPL), Version 1.0.
70+
71+
See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details.

‎cloud-infrastructure/storage/block-storage/asset/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Asset
2+
3+
OCI Block Storage assets.
4+
5+
6+
# License
7+
8+
Copyright (c) 2023 Oracle and/or its affiliates.
9+
10+
Licensed under the Universal Permissive License (UPL), Version 1.0.
11+
12+
See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details.

‎cloud-infrastructure/storage/block-storage/asset/block-volume-backup-and-restore.md

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# License
2+
3+
Copyright (c) 2023 Oracle and/or its affiliates.
4+
5+
Licensed under the Universal Permissive License (UPL), Version 1.0.
6+
7+
See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details.
8+
9+
10+
# Showcase manual block volume backup and restore
11+
12+
## Information
13+
- [Mounting an Object Storage Bucket as File System on Oracle Linux](https://blogs.oracle.com/cloud-infrastructure/post/mounting-an-object-storage-bucket-as-file-system-on-oracle-linux)
14+
- [OCI Compute - How to Install s3fs-fuse on Oracle Linux 8 (Doc ID 2938554.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2938554.1)
15+
16+
## Prepare your environment
17+
- set up your individual "operate" instance
18+
- An Oracle enterprise Linux server with Command Line Interface (*see [Set up your Server with Resilience by default using CLI](https://gitlab.com/hmielimo/cloud-resilience-by-default/#set-up-your-server-with-resilience-by-default-using-cli) for details*)
19+
- create and attach block volume: e.g. backup-test (*50GB*)
20+
- create a bucket e.g. backup-test
21+
22+
## Set needed variables (*please adjust variables to your needs before past into your terminal*)
23+
~~~
24+
DEVICEoPATH1=/dev/oracleoci/oraclevdc
25+
DEVICEoPATH2=/dev/oracleoci/oraclevdc1
26+
MYoSOURCE1=/dev/sdb
27+
MYoSOURCE2=/dev/sdb1
28+
DATAoDIR=/mnt/MyBlockVolume.backup-test
29+
OBJECTDATAoDIR=/mnt/MyObjectStorage.backup-test
30+
BUCKEToNAME=backup-test
31+
MYoREGION=<region>
32+
MYoNAMESPACE=<your_namespace>
33+
MYoSECREToKEY=<your_secret_key>
34+
MYoACCESSoKEY=<your_access_key>
35+
~~~
36+
37+
## Format and mount block volume
38+
~~~
39+
sudo umount ${DATAoDIR}
40+
sudo mkdir ${DATAoDIR}
41+
sudo fdisk -l
42+
sudo fdisk ${MYoSOURCE1}
43+
44+
n add a new partition
45+
p primary (0 primary, 0 extended, 4 free)
46+
Partition number (1-4, default 1):
47+
First sector (2048-419430399, default 2048):
48+
Last sector, +sectors or +size{K,M,G,T,P} (2048-419430399, default 419430399): +1G
49+
w write table to disk and exit
50+
51+
sudo mkfs -t ext4 ${DEVICEoPATH2}
52+
sudo mount ${DEVICEoPATH2} ${DATAoDIR}
53+
sudo chown -R opc:opc ${DATAoDIR}
54+
ls -lah ${DATAoDIR}
55+
sudo mount | grep ${DATAoDIR}
56+
lsblk
57+
~~~
58+
59+
## Update fstab
60+
~~~
61+
sudo vi /etc/fstab
62+
/dev/oracleoci/oraclevdc1 /mnt/MyBlockVolume.backup-test ext4 defaults,_netdev,noatime 0 2
63+
~~~
64+
65+
## Mounting an Object Storage Bucket as File System
66+
~~~
67+
# Generate Secret Key: <your_secret_key>
68+
69+
sudo yum update
70+
sudo dnf upgrade
71+
sudo dnf search epel
72+
73+
sudo dnf clean all
74+
sudo dnf repolist all
75+
sudo dnf install oracle-epel-release-el8
76+
77+
sudo tee /etc/yum.repos.d/ol8-epel.repo<<EOF
78+
[ol8_developer_EPEL]
79+
name= Oracle Linux \$releasever EPEL (\$basearch)
80+
baseurl=https://yum.oracle.com/repo/OracleLinux/OL8/developer/EPEL/\$basearch/
81+
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
82+
gpgcheck=1
83+
enabled=1
84+
EOF
85+
sudo dnf makecache
86+
sudo dnf repolist all
87+
88+
sudo yum install s3fs-fuse
89+
90+
echo ${MYoACCESSoKEY}:${MYoSECREToKEY} > ${HOME}/.passwd-s3fs
91+
chmod 600 ${HOME}/.passwd-s3fs
92+
cat ${HOME}/.passwd-s3fs
93+
sudo rm -fr ${OBJECTDATAoDIR}
94+
sudo mkdir ${OBJECTDATAoDIR}
95+
sudo chown -R opc:opc ${OBJECTDATAoDIR}
96+
sudo umount ${OBJECTDATAoDIR}
97+
s3fs ${BUCKEToNAME} ${OBJECTDATAoDIR} -o endpoint=${MYoREGION} -o passwd_file=${HOME}/.passwd-s3fs -o url=https://${MYoNAMESPACE}.compat.objectstorage.${MYoREGION}.oraclecloud.com/ -o nomultipart -o use_path_request_style
98+
ls -lah ${OBJECTDATAoDIR}
99+
sudo mount | grep ${OBJECTDATAoDIR}
100+
sudo mount | grep ${DATAoDIR}
101+
~~~
102+
103+
## Create a Set of test data
104+
~~~
105+
touch ${DATAoDIR}/test.tmp
106+
touch ${OBJECTDATAoDIR}/test.tmp
107+
108+
tee ${DATAoDIR}/test.txt<<EOF
109+
This is a testfile located in a block volume.
110+
EOF
111+
112+
tee ${OBJECTDATAoDIR}/test.txt<<EOF
113+
This is a testfile located in object storage.
114+
EOF
115+
116+
cat ${DATAoDIR}/test.txt
117+
cat ${OBJECTDATAoDIR}/test.txt
118+
119+
ls -lah ${DATAoDIR}
120+
ls -lah ${OBJECTDATAoDIR}
121+
~~~
122+
123+
## Backup block volume (*here only a partition for testing purposes*)
124+
~~~
125+
sudo umount ${DATAoDIR}
126+
sudo dd if=${MYoSOURCE} conv=sync,noerror bs=128M | gzip -c > ${OBJECTDATAoDIR}/my-disk.image.gz
127+
ls -lah ${OBJECTDATAoDIR}/my-disk.image.gz
128+
~~~
129+
130+
## Delete ${DATAoDIR}/test.txt
131+
~~~
132+
sudo mount ${DEVICEoPATH} ${DATAoDIR}
133+
ls -lah ${DATAoDIR}
134+
rm ${DATAoDIR}/test.txt
135+
ls -lah ${DATAoDIR}
136+
~~~
137+
138+
## Restore
139+
~~~
140+
sudo umount ${DATAoDIR}
141+
gunzip -c ${OBJECTDATAoDIR}/my-disk.image.gz | sudo dd of=${MYoSOURCE}
142+
~~~
143+
144+
## Test if restore was successful
145+
~~~
146+
sudo mount ${DEVICEoPATH} ${DATAoDIR}
147+
ls -lah ${DATAoDIR}
148+
~~~

‎cloud-infrastructure/storage/block-storage/asset/functions0.sh

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
#!/bin/bash
2+
# Version: @(#).functions0.sh 1.0.0
3+
# License
4+
# Copyright (c) 2023 Oracle and/or its affiliates.
5+
# Licensed under the Universal Permissive License (UPL), Version 1.0.
6+
# See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/folder-structure/LICENSE) for more details.
7+
#
8+
#@ functions needed to run healthcheck.storage.sh
9+
#@
10+
#
11+
# Update history:
12+
#
13+
# V 1.0.0 28.06.2023 initial version
14+
#
15+
16+
17+
if [ 1 -eq 1 ] ; then # define colors
18+
Color_Off='\e[0m' # Text Reset
19+
20+
# Regular Colors
21+
Black='\e[0;30m' # Black
22+
Red='\e[0;31m' # Red
23+
Green='\e[0;32m' # Green
24+
Yellow='\e[0;33m' # Yellow
25+
Blue='\e[0;34m' # Blue
26+
Purple='\e[0;35m' # Purple
27+
Cyan='\e[0;36m' # Cyan
28+
White='\e[0;37m' # White
29+
30+
# Bold
31+
BBlack='\e[1;30m' # Black
32+
BRed='\e[1;31m' # Red
33+
BGreen='\e[1;32m' # Green
34+
BYellow='\e[1;33m' # Yellow
35+
BBlue='\e[1;34m' # Blue
36+
BPurple='\e[1;35m' # Purple
37+
BCyan='\e[1;36m' # Cyan
38+
BWhite='\e[1;37m' # White
39+
40+
# Underline
41+
UBlack='\e[4;30m' # Black
42+
URed='\e[4;31m' # Red
43+
UGreen='\e[4;32m' # Green
44+
UYellow='\e[4;33m' # Yellow
45+
UBlue='\e[4;34m' # Blue
46+
UPurple='\e[4;35m' # Purple
47+
UCyan='\e[4;36m' # Cyan
48+
UWhite='\e[4;37m' # White
49+
50+
# Background
51+
On_Black='\e[40m' # Black
52+
On_Red='\e[41m' # Red
53+
On_Green='\e[42m' # Green
54+
On_Yellow='\e[43m' # Yellow
55+
On_Blue='\e[44m' # Blue
56+
On_Purple='\e[45m' # Purple
57+
On_Cyan='\e[46m' # Cyan
58+
On_White='\e[47m' # White
59+
60+
# High Intensity
61+
IBlack='\e[0;90m' # Black
62+
IRed='\e[0;91m' # Red
63+
IGreen='\e[0;92m' # Green
64+
IYellow='\e[0;93m' # Yellow
65+
IBlue='\e[0;94m' # Blue
66+
IPurple='\e[0;95m' # Purple
67+
ICyan='\e[0;96m' # Cyan
68+
IWhite='\e[0;97m' # White
69+
70+
# Bold High Intensity
71+
BIBlack='\e[1;90m' # Black
72+
BIRed='\e[1;91m' # Red
73+
BIGreen='\e[1;92m' # Green
74+
BIYellow='\e[1;93m' # Yellow
75+
BIBlue='\e[1;94m' # Blue
76+
BIPurple='\e[1;95m' # Purple
77+
BICyan='\e[1;96m' # Cyan
78+
BIWhite='\e[1;97m' # White
79+
80+
# High Intensity backgrounds
81+
On_IBlack='\e[0;100m' # Black
82+
On_IRed='\e[0;101m' # Red
83+
On_IGreen='\e[0;102m' # Green
84+
On_IYellow='\e[0;103m' # Yellow
85+
On_IBlue='\e[0;104m' # Blue
86+
On_IPurple='\e[0;105m' # Purple
87+
On_ICyan='\e[0;106m' # Cyan
88+
On_IWhite='\e[0;107m' # White
89+
fi
90+
91+
if [ 1 -eq 1 ] ; then # set environement
92+
. parameter1.sh
93+
fi
94+
95+
if [ 1 -eq 1 ] ; then # level 0 functions (this functions do not need/call other then level 0 functions)
96+
97+
function color_print() {
98+
if [ ${DEBUG_LEVEL} -eq 0 ] ; then echo -e "$1: $2 ${Color_Off}" ; fi
99+
if [ ${DEBUG_LEVEL} -eq 1 ] ; then echo -e "$1(${DEBUG_LEVEL}) $(date "+%d.%m.%Y %H:%M:%S") : $2 ${Color_Off}" ; fi
100+
if [ ${DEBUG_LEVEL} -eq 2 ] ; then echo -e "$1(${DEBUG_LEVEL}) $(date "+%d.%m.%Y %H:%M:%S") : $2 ${Color_Off}" ; fi
101+
if [ ${DEBUG_LEVEL} -gt 2 ] ; then echo -e "$1(${DEBUG_LEVEL}) $(date "+%d.%m.%Y %H:%M:%S") : $2 ${Color_Off}" ; fi
102+
}
103+
104+
TIMESTAMP=$(date "+%Y%m%d%H%M%S") # timestamp
105+
UNIQUE_ID="k4JgHrt${TIMESTAMP}" # generate a unique ID to tag resources created by this script
106+
if [ -e /dev/urandom ];then
107+
UNIQUE_ID=$(cat /dev/urandom|LC_CTYPE=C tr -dc "[:alnum:]"|fold -w 32|head -n 1)
108+
fi
109+
TMP_FILE_LIST=() # array keeps a list of temporary files to cleanup
110+
THIS_SCRIPT="$(basename ${BASH_SOURCE})" # sciptname
111+
112+
# Do cleanup
113+
function Cleanup() {
114+
color_print "${MYcolor}" "$PF1 function: $FUNCNAME"
115+
for i in "${!TMP_FILE_LIST[@]}"; do
116+
if [ -f "${TMP_FILE_LIST[$i]}" ]; then
117+
#echo -e "deleting ${TMP_FILE_LIST[$i]}"
118+
rm -f "${TMP_FILE_LIST[$i]}"
119+
fi
120+
done
121+
}
122+
123+
# Do cleanup, display error message and exit
124+
function Interrupt() {
125+
Cleanup
126+
exitcode=99
127+
echo -e "\nScript '${THIS_SCRIPT}' aborted by user. $Color_Off"
128+
exit $exitcode
129+
}
130+
131+
# trap ctrl-c and call interrupt()
132+
trap Interrupt INT
133+
# trap exit and call cleanup()
134+
trap Cleanup EXIT
135+
136+
function tempfile()
137+
{
138+
local __resultvar=$1
139+
local __tmp_file=$(mktemp -t ${THIS_SCRIPT}_tmp_file.XXXXXXXXXXX) || {
140+
echo -e "$Cyan ......#*** Creation of ${__tmp_file} failed $Color_Off";
141+
exit 1;
142+
}
143+
TMP_FILE_LIST+=("${__tmp_file}")
144+
if [[ "$__resultvar" ]]; then
145+
eval $__resultvar="'$__tmp_file'"
146+
else
147+
echo -e "$Cyan ......#$__tmp_file"
148+
fi
149+
}
150+
151+
fi
152+
153+
# ---------------------------------------------------------------------------------------------------------------------------------------------
154+
# end of file
155+
# ---------------------------------------------------------------------------------------------------------------------------------------------

0 commit comments

Comments
 (0)