Merge pull request #49 from oracle-devrel/develop

Automation Toolkit Release v11

Author: xs2suruchi

README.md

@@ -29,6 +29,7 @@
   <li> <a href="/cd3_automation_toolkit/documentation/user_guide/GreenField.md">Green Field Tenancies</a>
     <ul>
       <li> <a href="/cd3_automation_toolkit/documentation/user_guide/NetworkingScenariosGF.md"</a><b> Must Read :</b> Managing Network for Greenfield Tenancies</li>
+      <li> <a href="/cd3_automation_toolkit/documentation/user_guide/RemoteExecution.md"</a><b> Must Read :</b> Remote Execution/Cloud Init for OCI compute instances</li>
     </ul>
     </ul>
   </li>
@@ -73,7 +74,7 @@
 CD3 stands for <b>C</b>loud <b>D</b>eployment <b>D</b>esign <b>D</b>eliverable.
 The CD3 Automation toolkit has been developed to help in automating the OCI resource object management. 
 <br><br>
-It reads input data in the form of CD3 Excel sheet and generates the terraform files which can be used to provision the resources in OCI instead of handling the task through the OCI console manually. The toolkit also reverse engineers the components in OCI back to the Excel sheet and Terraform configuration. This generated Terraform code can be used by the OCI Resource Manager or leveraged by the organisations’ CI/CD processes.
+It reads input data in the form of CD3 Excel sheet and generates Terraform files which can be used to provision the resources in OCI instead of handling the task through the OCI console manually. The toolkit also reverse engineers the components in OCI back to the Excel sheet and Terraform configuration. This generated Terraform code can be used by the OCI Resource Manager or leveraged by the organisations’ CI/CD processes.
 <br><br>
 <kbd>
 <img width="748" alt="Screenshot 2022-12-30 at 11 57 41 AM" src="https://user-images.githubusercontent.com/111430850/210614513-5d2e97a6-3c1e-4a2b-a793-3a1b6410c856.png">
@@ -84,15 +85,18 @@ It reads input data in the form of CD3 Excel sheet and generates the terraform f
 
 | OCI Services | Details |
 | --------- | ----------- |
-| [IAM/Identity](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#iamidentity) | Compartments, Groups, Dynamic Groups, Policies |
-| [Network](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#network) | VCNs, Subnets, DRGs, IGWs, NGWs, LPGs, Route Tables, DRG Route, Tables, Security Lists, Network Security Groups, Application Load Balancer, Network Load Balancers |
+| [IAM/Identity](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#iamidentity) | Compartments, Groups, Dynamic Groups, Policies, Users, Network Sources |
 | [Governance](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#governance) | Tags (Namespaces, Tag Keys, Defined Tags, Default Tags, Cost Tracking) |
-| [Compute](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#compute) | Instances – VM, BM, Dedicated VM Hosts |
-| [Storage](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#storage) | FSS, Block and Boot Volumes, Backup Policies |
+| [Network](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#network) | VCNs, Subnets, VLANs, DRGs, IGWs, NGWs, LPGs, Route Tables, DRG Route, Tables, Security Lists, Network Security Groups, Remote Peering Connections, Application Load Balancer, Network Load Balancers |
+| [DNS Management](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#private-dns)                                       | Private DNS - Views, Zones, rrsets/records and Resolvers  |
+| [Compute](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#compute) | Instances supporting Market Place Images, Remote Exec, Cloud-Init scripts, Dedicated VM Hosts |
+| [Storage](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#storage) | FSS, Block and Boot Volumes, Backup Policies, Object Storage Buckets and logging for write events |
 | [Database](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#database) | Exa Infra, ExaCS, DB Systems VM and BM, ATP, ADW |
 | [Management Services](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#management-services) | Events, Notifications, Alarms, Service Connector Hub (SCH) |
 | [Developer Services](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#developer-services) | Resource Manager, Oracle Kubernetes Engine (OKE) |
-| [CIS Landing Zone Compliance](/cd3_automation_toolkit/documentation/user_guide/learn_more/CISFeatures.md#additional-cis-compliance-features) | Download and Execute CIS Compliance Check Script, VCN Flow Logs, Cloud Guard, Object Storage, Key Vault, Budget |
+| [Logging Services](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#logging-Services) | VCN Flow Logs, LBaaS access and error Logs, OSS Buckets write Logs |
+| [SDDCs ](/cd3_automation_toolkit/documentation/user_guide/learn_more/CD3ExcelTabs.md#sddcs-tab) | Oracle Cloud VMWare Solutions |
+| [CIS Landing Zone Compliance](/cd3_automation_toolkit/documentation/user_guide/learn_more/CISFeatures.md#additional-cis-compliance-features) | Download and Execute CIS Compliance Check Script, Cloud Guard, Key Vault, Budget |
 
 
 [Click here](/cd3_automation_toolkit/documentation/user_guide/prerequisites.md) to get started and manage your OCI Infra!

cd3_automation_toolkit/Compute/create_terraform_instances.py

@@ -24,14 +24,15 @@ def parse_args():
     parser = argparse.ArgumentParser(description='Creates Instances TF file')
     parser.add_argument('inputfile', help='Full Path of input CD3 excel file')
     parser.add_argument('outdir', help='Output directory for creation of TF files')
-    parser.add_argument("service_dir",help="subdirectory under region directory in case of separate out directory structure")
+    parser.add_argument("service_dir",
+                        help="subdirectory under region directory in case of separate out directory structure")
     parser.add_argument('prefix', help='TF files prefix')
     parser.add_argument('--config', default=DEFAULT_LOCATION, help='Config file name')
     return parser.parse_args()
 
 
 # If input is CD3 excel file
-def create_terraform_instances(inputfile, outdir, service_dir,prefix, config):
+def create_terraform_instances(inputfile, outdir, service_dir, prefix, config):
     boot_policy_tfStr = {}
     tfStr = {}
     ADS = ["AD1", "AD2", "AD3"]
@@ -60,7 +61,7 @@ def create_terraform_instances(inputfile, outdir, service_dir,prefix, config):
     # Take backup of files
     for eachregion in ct.all_regions:
         resource = sheetName.lower()
-        srcdir = outdir + "/" + eachregion+ "/" + service_dir + "/"
+        srcdir = outdir + "/" + eachregion + "/" + service_dir + "/"
         commonTools.backup_file(srcdir, resource, auto_tfvars_filename)
         tfStr[eachregion] = ''
         boot_policy_tfStr[eachregion] = ''
@@ -143,10 +144,12 @@ def create_terraform_instances(inputfile, outdir, service_dir,prefix, config):
                         vcn_name = subnets.vcn_subnet_map[key][1]
                         subnet_id = subnets.vcn_subnet_map[key][2]
                     except Exception as e:
-                        print("Invalid Subnet Name specified for row " + str(i + 3) + ". It Doesnt exist in Subnets sheet. Exiting!!!")
+                        print("Invalid Subnet Name specified for row " + str(
+                            i + 3) + ". It Doesnt exist in Subnets sheet. Exiting!!!")
                         exit()
 
-                tempdict = {'network_compartment_id': commonTools.check_tf_variable(network_compartment_id), 'vcn_name': vcn_name,
+                tempdict = {'network_compartment_id': commonTools.check_tf_variable(network_compartment_id),
+                            'vcn_name': vcn_name,
                             'subnet_id': subnet_id}
 
             if columnname == 'Display Name':
@@ -159,6 +162,15 @@ def create_terraform_instances(inputfile, outdir, service_dir,prefix, config):
                 compartment_var_name = commonTools.check_tf_variable(compartment_var_name)
                 tempdict = {'compartment_tf_name': compartment_var_name}
 
+            if columnname == 'Remote Execute':
+                if columnvalue != "":
+                    if ("@" in columnvalue):
+                        remote_execute = columnvalue.strip().split("@")
+                        tempdict = {'remote_execute': remote_execute[1],
+                                'bastion_ip': remote_execute[0]}
+                    else:
+                        tempdict = {'remote_execute': columnvalue.strip() }
+
             if columnname == 'Custom Policy Compartment Name':
                 if columnvalue != "":
                     custom_policy_compartment_name = columnvalue.strip()
@@ -196,7 +208,7 @@ def create_terraform_instances(inputfile, outdir, service_dir,prefix, config):
             if columnname == "SSH Key Var Name":
                 if columnvalue.strip() != '' and columnvalue.strip().lower() != 'nan':
                     if "ssh-rsa" in columnvalue.strip():
-                        ssh_key_var_name = "\"" + columnvalue.strip() + "\""
+                        ssh_key_var_name = columnvalue.strip()
                     else:
                         ssh_key_var_name = columnvalue.strip()
                     tempdict = {'ssh_key_var_name': ssh_key_var_name}

cd3_automation_toolkit/Compute/getInitUserCredsforWindows.py

@@ -32,6 +32,18 @@ instances = {
             fault_domain         = "{{ fault_domain }}"
             {% endif %}
 
+            {% if remote_execute %}
+            remote_execute       = "{{ remote_execute }}"
+            {% endif %}
+
+            {% if bastion_ip %}
+            bastion_ip           = "{{ bastion_ip }}"
+            {% endif %}
+
+            {% if cloud_init_script %}
+            cloud_init_script    = "{{ cloud_init_script }}"
+            {% endif %}
+
             {% if dedicated_vm_host_tf %}
             dedicated_vm_host_id = "{{ dedicated_vm_host_tf }}"
             {% endif %}
@@ -52,7 +64,7 @@ instances = {
             {% endif %}
 
             {% if hostname_label %}
-            hostname_label         = "{{ hostname_label }}"
+            hostname_label   = "{{ hostname_label }}"
             {% endif %}
 
             {% if nsg_ids %}
@@ -64,7 +76,7 @@ instances = {
             {% endif %}
 
             {% if memory_in_gbs %}
-            memory_in_gbs = {{ memory_in_gbs }}
+            memory_in_gbs    = {{ memory_in_gbs }}
             {% endif %}
 
             {% if capacity_reservation_id %}

cd3_automation_toolkit/Compute/templates/instances-template

@@ -24,9 +24,7 @@
 
 
 def print_adbs(region, vnc_client, adb, values_for_column, ntk_compartment_name):
-    # def print_exa_infra(region, exa_infra, values_for_column, ntk_compartment_name):
     adb_tf_name = commonTools.check_tf_variable(adb.display_name)
-    # maintenance_window = exa_infra.maintenance_window
 
     adb_subnet_id = adb.subnet_id
 
@@ -56,7 +54,6 @@ def print_adbs(region, vnc_client, adb, values_for_column, ntk_compartment_name)
             whitelisted_ips = whitelisted_ips + "," + ip
 
         whitelisted_ips = whitelisted_ips[1:]
-    # importCommands[region.lower()].write("\nterraform import oci_database_cloud_exadata_infrastructure." + exa_infra_tf_name + " " + str(exa_infra.id))
     importCommands[region.lower()].write(
         "\nterraform import \"module.adb[\\\"" + adb_tf_name + "\\\"].oci_database_autonomous_database.autonomous_database\" " + str(adb.id))