...

1. ONAP release must be readily deploy-able, to allow a test framework to run on the deployment and test a VNF.  
2. ONAP deployment must be reproducible to ensure VNF tests are conducted in a uniform environment.
3. Definition of life cycle requirements (i.e. what the test cases validate)
   1. Both HEAT & TOSCA 

      1. What does a mean (formally for a test case requirement) to "instantiate" (or startup) a VNF?

         1. 
            

            Info
            from Victor, Definition of VNF onboarding for TOSCA based VNF. Verify the VNF package by using VNFSDK compliance check test cases. upload a VNF package to ONAP system verify the VNF Package already exist on the ONAP system.

            
            

         2. 
            

            Info

            from Victor, Definition of VNF instantiation for TOSCA based VNF. Trigger an "instantiate VNF" operation to Controller verify that the requested grant for the "instantiate VNF" operation has been approved by the controller VNF related software images have been successfully added to the image repository managed by the VIM or already exist on the VIM system(Currently, ONAP is manually doing this.) Verify that the requested virtualised resources have been allocated by the VIM according to the VNFD Verify that virtualised resource allocation constraints have been met by querying the VIM Verify that any existing virtualised resources have not been affected by the allocation of the new virtualised resources by querying the VIM Verify that the VNF instance resources are visible on the controller

            
            

      2. Does instantiation include configuration?
      1. Victor: both heat and Tosca template could include the inject-file/user-data(cloud-init) as the day0 configuration file of a VNF.  
      2. Trevor: I would argue against configuration for the MVP phase.  There are a variety of options to execution post-instantiation configuration management (Ansible, Chef, etc.).   Since there are multiple options for a compliant VNF to utilize it would not make sense to offer only a subset of options, and offering any option would involve setup, configuration, and execution of the configuration mgmt option itself. 
      3. Does instantiation include health check?
         1. Victor: Could be yes, we could use the query interface (query the VNF detail info) to implement the health-check func.
      4. HEAT
         1. Trevor Lovett & Ryan Hallahan will help with this.
         2. The following is a DRAFT - needs further review and input
         3. Prerequisite Decision Points 
            1. Are we planning for decentralized (VNF Provider must setup their own ONAP instance and peform their own testing) or centralized testing (VNF Provider is submitting their VNF and associated settings for testing)  model?  Assuming the former, but need confirmation as the later has additional automation and other concerns to care for (network creation, image registration, etc.)
         4. Assumptions 
            1. Test scripts will utilize a default license model in SDC
            2. Compliance with test suite assumes the VNF has already has compliant packaging certified
            3. If utilizing full automated testing, then assume the Service model will only include a single VF for instantiation.  It would likely be too difficult to automatically create a multi-VF service in the first phase, and not necessary for the scope of testing
            4. Test Input will include the preload information in some TBD file format that can be loaded into SDNC.
         5. Implementation Options 
            1. Within ONAP today, the Integration Testsuite generally has all the building blocks to automate and execute the steps below.   It would need some modification to handle a more arbitrary VNF as it is not used in this way today, but based on preliminary analysis this looks achievable. 
            2. Orange Python Framework - This is another option that can be considered.  It looks to handle the instantiation use case, and potentially others.  
         6. Potential Test Flow
            1. Intialize Vendor and Category Information
            2. Create the VSP in SDC
            3. Upload Heat Archive
            4. (Optional) Assign any Unassigned Files to Artifacts
            5. Validate the VSP and ensure now Errors exist (warnings are OK)
            6. Assign the Vendor License Model to the VSP (assumes a single VLM for testing purposes)
            7. Create the Virtual Function
               1. Import the VSP (find using Name or ID from prior steps)
               2. Set name of VF (auto-assign or make input into test script), contact and other required fields
            8. Create Service
               1. Set Name (auto-assign based on VSP or make input into test script)
               2. Assign required or optional fields based on test script input
               3. Assign VF to the Servce Model
            9. Distribute the Service Model and validate successful Distribution
            10. Submit Preloads to SDNC
            11. Trigger Instantiation of Base Module from VID (NOTE: Need to see how we handle multi-module VNFs - presumably we can query this information and instantiate each individully)
            12. Verify sucessfull instantiation
            13. Health-check TBD - needs further discussion
      5. TOSCA
         1. Weitao Gao will help with this.

         2. 
            

            Info

            from Victor, Definition of VNF instantiation for TOSCA based VNF. Trigger an "instantiate VNF" operation to Controller verify that the requested grant for the "instantiate VNF" operation has been approved by the controller VNF related software images have been successfully added to the image repository managed by the VIM or already exist on the VIM system(Currently, ONAP is manually doing this.) Verify that the requested virtualised resources have been allocated by the VIM according to the VNFD Verify that virtualised resource allocation constraints have been met by querying the VIM Verify that any existing virtualised resources have not been affected by the allocation of the new virtualised resources by querying the VIM Verify that the VNF instance resources are visible on the controller

            
            

   2. Test case description template for specifying VNF validation test purpose, implementation steps and pass/fail criteria.
   3. Definition of the set of ONAP components and their configuration required for the testing (ONAP profile used for testing).
   4. Definition of test infrastructure requirements needed for testing (i.e. hardware with compute / network / storage and pod requirements to run testing).
   5. Test tooling that drives testing through existing ONAP interfaces.
      1. HEAT
      2. TOSCA

What is needed / Work to do Matrix

Test Life Cycle

Items to do 

Open Questions

1. Can the test requirements or definitions (procedures) by pulled from, or reuse, the ETSI TST-0007
   1. What are the integration and testing interfaces that are currently available, i.e. used by the integration team / gating team?

...