Overview

This page is intended to capture discussions and work relating to the longer term strategy and direction of testing within the LFN programs.

Testing Scopes

Note, the following is a work in progress.

Compliance (to a specification or standard):

VNF Compliance

• Conformance to HEAT or TOSCA Templates, as defined by ONAP Requirements.

• Conformance to ONAP VNFRqmts documentation which will integrate other SDO requirements (ie, 5G, BBF, ETSI,...)

• Documentation of VNF is available.

NFVI (Doesn't this include VIM?) Compliance:

•  ETSI ISG NFV interface standards
• should this be expanded to whitebox servers, DPDK,  FDIO,...?

Validation:

• VNF Validation:

  • On-boarding: Validate the VNF Artifact(s) is able to be "consumed" by an ONAP instance.

    • Open questions: 1) What are the requirements defined for "On-boarding?" What is the requirement for the "ONAP instance?" 3) Is there a requirement for backwards compatibility? 4) Besides ONAP are there other organizations defining this?

  • Instantiation: Validate the VNF can be "launched" by an ONAP instance on top of an NFVI.

    • Open questions: 1) What is the requirements for ONAP Instance? 2) What is the requirements for the NFVI? 3) What are the requirements for the "VNF launch?"

  • Validate various life-cycle operations are possible with the VNF running on the ONAP Instance.

    • Open questions: 1) What the required "life-cycle operations?" 2) What the requirements for each "operation?" 3) What are the requirements for the "ONAP instance?"

    • Life Cycle Operations  of VNF (long term vision) - my version of FCAPS:
    • Fault Management
    • Capacity Management
    • Billing/Accounting Management
    • Performance Management
    • Security Management 
• NFVI Validation:
  • Instantiate NFVI/VIM
  • Life Cycle of NFVI/VIM - Healthcheck?

Performance:

• NFVI Performance

  • Capacity: of RAM, CPU, storage

  • Stability:

    • Chaos monkey tests?
  • data throughput, CPU performance, RAM/Storage access performance?
  • What is my reference implementation of NFVI? GSMA Profiles (B, N, and C). ETSI ISG NFV has a performance working group having specs. Led by Al Morton, AT&T. Also I think Pierre Lynch, IXIA, leads the testing spec group also.
  • What are the metrics we need to measure? (align with ETSI, GSMA defined metrics).
    • 1 test per metric?
• VNF Performance.
  • huge dependency on NFVI. need reference NFVI to compare against. (GSMA Profiles).
  • what are the metrics we need to measure? (align with ETSI, GSMA defined metrics).
    • 1 test per metric?
  • VNF Reference Implementation. 

Security:

• Open question: Where do these requirements come from? Who is responsible for the test cases?

Slide presented/created in Paris 2019 F2f