Topic Leader(s)

Topic Overview

60 minutes, Deepak Kataria 

BluVal (Blueprint Validation) Framework is a cloud native full stack test automation tool based on a disaggregated and layered approach for test and validation of implementations to ensure conformance. BluVal disaggregates the implementation and associated manifests into their layers and provides automated validation framework to test each layer one by one to achieve end-to-end validation of the implementation. All results are standardized by using Robot Framework which creates an html file of test results that is easy to read and interpret. By virtue of its modular design, the BluVal framework is extensible and provides a simple, declarative test harness for authoring and executing test suites. New functional tests can be written up from scratch in Robot, with simple human-readable commands, or existing upstream tests can be quickly integrated with a few lines of template code that can call any other testing or scripting tool. BluVal framework also provides a portal (User Interface) for viewing executed test result.

Slides & Recording

Slides presented, with live interactive demo, and Q&A.

Cloud-Native Full Stack Conformance Validation Framework.mp4

Agenda

Cloud-Native Full Stack Conformance Validation

  • Presentation
  • Demo
  • Q&A
  • Next Steps
  • Attendees

Minutes

Presentation:  Refer to slides for details of the following shared:

  • Deepak presented an overview of the BluVal (Blueprint Validation) Framework, demonstrating a cloud-native full stack test automation tool based on a disaggregated and layered approach for test and validation of cloud implementations to ensure conformance.
  • Several existing use-cases shared where BluVal is used for validations today: e.g. Radio Edge Cloud, Connected Vehicle, Integrated Clud Native (ICN), etc.
  • End-to-End automated validation elements comprised of Layers of CD testing from Hardware and OS, to K8S, OpenStack, and Workloads.
  • CI/CD operates as a standalone, or integrated LF Edge networks for ingesting blueprints for validation, leveraging Jenkins to execute and publish results to a database, viewable from a User Interface (UI).
  • Value-add discussed for layered approached, allowing users to skip layers if not ready/desired, including performing pre-production workload tests from Unit, to Pre-Prod, to Production deployments.
  • Platform is versatile and robust to ingest Opensource test-frameworks and harnesses.
  • Evolutionary roadmap includes integration of Jenkins console for monitoring build executions and results.

Demo

  • Demo provided of the UI navigation, Registering a Lab, Selecting a Lab, and viewing Results.
  • Reviewed capability to view results, and drill down into the specific layer executed and failures observed.

Q&A

  1. From Gergely Csatari 
    • Q: Are these RA2 compliant Edge Cloud Deployments?
    • A:  The Edge Cloud deployments shown in the PPT are 'RA2-like', leveraging kubernetes, but have not been tested with RC2 requirements and tests.  Future scope includes integrated RC2 scope into BluVal.

  2. From Saad Sheikh 

    • Q: Have we tested some use cases E2E using this (BluVal) like 5G Core Infrastructure at Edge etc?

    • A: Yes, Radio Edge Cloud blueprints have been tested, which are part of the Telco appliance blueprint family, and Unicycle OVS-DPDK part of Network Cloud blueprint family.  Refer to slides for more examples (pp 3-4).

  3. From Cedric Ollivier

    • Q: Does BluVal use the OPNFV/OVP Database to store results, or another database?

    • A: It’s another (separate) opensource LF Edge database.

  4. From Saad Sheikh 

    • Q: Kindly can you share detail URL to know about Dis-agg testing details and if possible do you have some results ?

    • A: Refer to slides for details (pp 10,14).
  5. From Mike Fix
    • Q: Are there any known limitations with the number of layers which can be utilized?
    • A: Layers are completely customizable, can be added/removed as needed per blueprint needs, and there are no limitations to the number of layers which can be added and executed.
  6. From Cedric Ollivier
    • Q: What framework is used for executing and disaggregating testing (e.g. CI, CD)?
    • A: Jenkins and Robot are used either locally (i.e. independent CI), or within an LF-type deployment CI pipeline for blueprint executions, pushing results, and disaggregating results into the different layers in the database. 

Next Steps

Consensus reached by community / attendees that there is value-add to extend the integration of RC2 requirements for Anuket into BluVal, and provides results and observations from that integrated validation.  

More specifically, Deepak Katariato (all opened 6/8 with presentation):

  • Partner with Anuket Members, and respective TSC(s), to integrate RC2 into BluVal.

  • Review RC2 BluVal results, observations, and key takeaways with Anuket RI2/RC2 community.

  • Verify results align with CNCF requirements and compare with existing benchmark results, if available.

  • Update BluVal blueprints with RC2 requirements, and ensure alignment with the LF Edge and Anuket communities. 
  • Provide answers or clarity, to questions needed above (#1, #4) (Closed 6/8)

Attendees

  1. Deepak Kataria(Presenter)
  2. MICHAEL FIX
  3. Jim Baker
  4. Cedric Ollivier
  5. Gergely Csatari
  6. Al Morton
  7. Aneesh Kumar 
  8. Bala Gopala Krishna Ramanadam
  9. Georg Kunz
  10. Gurpreet Singh
  11. Hammad Zafar
  12. Ildiko Vancsa
  13. Loakeim Samaras
  14. Jie Niu
  15. Kevin Timoney
  16. Lincoln Lavoie
  17. Manoj Mourya
  18. Michael Pedersen
  19. Pankaj Goyal
  20. Quang-Huy Nguyen
  21. Rihab Banday
  22. Rohit Kumar
  23. Saad Sheikh
  24. Scot Steele
  25. Sirisha Gopigiri 
  26. Sridhar Rao
  27. Ulrich Kleber
  28. Walter Kozlowski
  29. Mallory

Action Items

  • See Next Steps above