README

• This TOC page has been created with Ahmed El Sawaf Karine Sevilla Petar Torre Saad Sheikh Tomas Fredberg Walter Kozlowski and Pankaj Goyal as editors/owners

Action Items

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

9.5 Automation

9.5.1 Cloud Infrastructure LCM Automation ( including Platform Software and CI/CD)  

End-2-End understanding of what type of automation ought to be done on what layers and includes

9.5.1.1. hardware configuration CI/CD

• Example, BIOS settings, Reset, Power Modes,
• HW component discovery and status/health supervision
• Composability of physical hardware resources (components)
• Hardware Accelerator discovery, loading and assignment
• Firmware Update and supervision (measurement) of Signed FW on known Authentic HW
• Security related certificates, keys and rolesHardware Accelerators

9.5.1.2. networking automation

• Physical network cabling detection and supervision
• L2, L3 and QoS Automation of Server NIC Network Assignment and Switch Fabric Network Assignment based on HW Provisioning in the HW Infrastructure Layer
• Partitioning, provisioning, enforcement of Overlay switching shared resources e.g. VLAN and VxLAN

9.5.1.3. software development CI/CD for infrastructure (not workloads) – Saad

what are the components of "infrastructure software": provisioning and configuration of the infrastructure (servers, network and storage; operations and management software)

Telco DevOps is still not mature within context of Telco infrastructure and service primarily due to reason that software in Telecom is primarily provided by vendors with operators almost no clear visibility about the software constructs itself , it means only Infrastructure and Networking Automation can not automate complete service .  

End to End specification of how to

...

deliver automation in a hybrid Infrastructure is required with following features

• CI (Continuous integration) for Telco service 

                a)  When vendor will update new feature or new code files ( Packages , Helm charts etc) how it will be merged with existing software

                b)  Track and fix bugs of incremental software running on Infrastructure

• CD (Continuous Delivery) for Telco service 

                 a) Testing of new code base e.g Test Bench , vSPerf etc

                 b) Automation of Code base with Infrastructure e.g Python , GNPy , Heat etc

                 c) Produce the software artifacts which will be deployed on Infrastructure , example is Sol4 packages , Sol1 templates 

                 d) Testing of packages including Infrastructure DevOps

                 e) Output and benchmark the packages in staging Environment  , key example here is Telco PaaS capability

• CD (Continuous Deployment) for Telco service 

                 a)  Software onboarding in the Infrastructure it involves all deployments in both CNTT RM infrastructure and related VNF/CNF codes/configurations including simulation and Test execution environment

                 b) CI/CD pipeline , when a new VNF/CNF code need onboard or just a simple capacity expansion is required , It is not just about Infrastructure and Networking Automation but E2E automation including VNF/CNF part 

                 c)  Here the Key requirement is once a new Release is output all the process from build/test/validate and onboard is  done automatically 

                 d) Handover points between Infrastructure and Orchestration need to be validated and verifiable in the form of SLA/KPI's 

I think if all team agree then we can output DevOps Reference Architecture after team consensus covering Infra 

Lessons Learnt from ETSI NOC and CNCF conformance:

Following two domains we must catch for E2E definition and service 

                 →  CI explaining pipeline view between Lab , Staging and Production

                 →  CT , the testing of service 

                 →  Pipeline architecture e.g  Jira , Jenkins and Necessary SDK's in Orchestration through which we can automate E2E including Infrastructure

CI/CD requirements

9.5.1.4. software deployment CI/CD (operator environment) – covered in 9.5.2.2 (inserted 2020-12-07)

9.5.1.5. Identify "Closed Loop Automation" as a Gap in Ch10 – Done (PR #2123 -need reviews)

9.5.2 

...

Software onboarding automation (including CI/CD). Owner: Walter Kozlowski

9.5.2.1.

...

Software onboarding automation - the scope for RM is to describe support only but leave the details to RA/RI; Walter Kozlowski to open PR

The Cloud Infrastructure workload onboarding process describes activities needed for the integration of tenants' workloads into the Cloud Infrastructure environment. Typically, this business process consists of the following key phases:

1.  Tenant Engagement and Workload Evaluation:
   1.  In this phase the request from the tenant to host a workload on the Cloud Infrastructure platform is assessed and a decision made on whether to proceed with the hosting request.
   2. This phase may also involve the tenant accessing a pre-staging environment to perform their own evaluation and/or pre-staging activities in preparation for later onboarding phases.
2. Workload Packaging:
   1.  The main outcome of this phase is to produce the workload deployable image and  the deployment manifests (such as TOSCA blueprints or HEAT templates or Helm charts) that will define the Cloud Infrastructure service attributes for the workload. 
   2. The workload packaging can be performed by the tenant, through self-service capabilities or by the Cloud Infrastructure Operations team.
3. Workload Validation and Certification:
   1. In this phase the workload is deployed and tested to validate it against the service design and other Operator specific acceptance criteria, as required.
   2. Workload validation and certification should be automated using CI/CD toolsets / pipelines and Test as a Service (TaaS) capabilities.
4. Publish Workload:
   1. After the workload is certified the final onboarding process phase is for it to be published to the Cloud Infrastructure  production catalogue from where it can be instantiated on the Cloud Infrastructure platform by the tenant.

All phases described above can be automated using technology specific toolsets and procedures.  Hence, details of such automation are left for the technology specific Reference Architecture and Reference Implementation specifications.

9.5.2.2. Software CI/CD Requirements Pankaj Goyal to open PR

The requirements including for CI/CD for ensuring software security scans, image integrity checks, OS version checks, etc. prior to deployment, are listed in the Table XX.XX (below). Please note that the tenant processes for application LCM (such as updates) are out of scope. For the purpose of these requirements, CI includes Continuous Delivery, and CD refers to Continuous Deployment.

...

9.5.3.1. requirements for enterprise processes prior to tenant creation on the platform

Here is a starting set – Maybe too low level; requirements can be created once we agree on the correct set:

• Validate that the Capacity can satisfy the tenant requested quota for vCPU, RAM, Disk, Network Bandwidth
• Validate that the Cloud Infrastructure can meet tenant's performance requirements (e.g. I/O, latency, jitter, etc)
• Validate that the Cloud Infrastructure can meet tenant's resilience requirements
• Validate any requested private flavours
• For VM-based environments:
  • Verify that any requested private flavours have been created
  • Verify that the metadata for these private flavours have been created
  • Verify that the tenant has permissions to use the requested private flavours
  • Validate that host aggregates are available for specified flavors (public and private)
  • Verify that the metadata matches for the requested new flavours and host aggregates
• Verify that the networks requested by the tenant exist
• Verify the metadata: 1. Keypairs must be higher than default 2. Networks must be higher than default
• Add all Tenant Members and configure their assigned roles in the Enterprise Identity and Access management system (e.g., LDAP)
• Create Tenant
• Using a proto- or Tenant provided HEAT-template/Helm-chart for a NF and perform sanity test (e.g., using scripts test creation of VM/container, ping test, etc.)
• Verify and Validate Tenant Images: virus scan, correct OS version and patch, etc.

9.5.3.2. tenant networking automation

...