DRAFT (Table to be completed)
| Testsuite | VTP | Onap-tests | ... | ||
|---|---|---|---|---|---|
| Project | repository | https://git.onap.org/testsuite/ | https://git.onap.org/vnfsdk/refrepo/ | https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests | |
| Leader | ONAP Integration team (Brian, Yang, Gary, Marco, ..) | VNFSDK (Victor, Kanag) | Orange | ||
| Stats (February - may 2019) | 27 contributors / 95 patches / more than 5 different companies) | 3 contributors / 20 patches / 2 companies | 9 contributors / 145 patches/ 3 companies | ||
| Language | RobotFramework, Python | Java. Also supports to run test cases implemented in different languages | Python | ||
| Links | https://wiki.onap.org/display/DW/Robot+Framework+Development+Guide https://wiki.onap.org/display/DW/Robot+Healthcheck+Tests+on+ONAP+Components | https://wiki.onap.org/pages/viewpage.action?pageId=43386304 | https://wiki.onap.org/pages/viewpage.action?pageId=6593670&preview=%2F6593670%2F54722733%2Fonap_tests.pdf | ||
| Short Description | Robotframework testsuites to perform all types of tests: healthcheck, CSIT, VNF, web interface |
| Python SDK for ONAP to manage VNF Life cycle based on VVP compliant Heat descriptor. | ||
| Current Usage in ONAP |
| ||||
|
|
| ||||
| VNF use cases currently supported |
|
|
| ||
| Integrations | |||||
VTP provides a framework to integrate any testtools/test cases with it, so it can be executed from OVP portals, CI/CD, SDC, etc. Today VTP is integrated as below and can be extented to integrated with any tools
VTP ↔ SDC
VTP ↔ Dovetail
VTP → VVP
VTP → 3rd Party testing tools
Basically it unifies the test cases used with ONAP. and test cases could vary based on its requiredment such as LCM, compliance.| Other ONAP components | natively integrated in onap namespace (robot pod) | Yes integrated with SDC | Black box approach versus ONAP - separation between SUT and the test tool, no hardcoded values in ONAP configurations embedded in xtesting framework (used also for infrastructure tests - same DB/API used by Dovetail for infra verification) Infra results: http://testresults.opnfv.org/test/api/v1/results?case_name=tempest_full&last=1 vnf results with ONAP: http://testresults.opnfv.org/onap/api/v1/results?case_name=basic_vm&last=1 | ||
| VVP | Yes | Yes consider VVP for heat template verification on code linting as a linter on patch submission (check all the compliancy of all refrenced heat files) based on ONAP official VVP docker | |||
| Dovetail/OVP | Yes | No not natively but reuse all the same building block used by Dovetail (DB, test API, dockerization) | |||
| Item to do based on MVP page | Automation Script(s) to on-board VSP | Yes | Yes |
| Yes | ||
| Automation Script(s) to Create VF | Yes |
Yes | Yes | |
| Automation Script(s) to Create Service | Yes | Yes |
| Yes | ||
Automation Script(s) to Pre-Load Image | Yes | Yes |
NOT an ONAP operations, and is NFVI action, will use dovetail test case | Yes | |
| Automation Script(s) to Submit Preloads | Yes | Yes |
| yes | ||
| Automation Script(s) to Instantiate VNF | Yes | Yes |
| Yes | ||||
| Automation Script(s) to Healthcheck VNF | Yes | Yes | Yes | |
Automation to create OpenStack resources (via openstacksdk/shade) | Yes -NOT an ONAP operations, and is NFVI action, will use dovetail test case, -This could be managed thru MultiVIM test cases if required other than one listed https://wiki.onap.org/display/DW/OVP+LCM+Support | Yes | ||
| Automation to clean OpenStack resources | Yes -NOT an ONAP operations, and is NFVI action, will use dovetail test case -This could be managed thru MultiVIM test cases if required other than one listed https://wiki.onap.org/display/DW/OVP+LCM+Support | Yes | ||
| Automation to clean ONAP resources (Module, VNF, service instance) | Yes |
| Yes | ||
| Link with VNFREQTS |
| Yes | Yes | No | |||
| Descriptors | Tosca support | Yes | No but reuse of Tosca artifacts provided by SDC for runtime operations | ||
| Heat support | Yes |
| Yes |
| Yes |
Yes
as a linter on patch submission (check all the compliancy of all refrenced heat files) based on ONAP official VVP docker
| CI/CD integration | NO but Provides API, which could be integrated with CI//CD where ever requried | ||
Yes xtesting-onap with gitlabci for VVP verification on linting + chained-ci for On demand ONAP deployment + test | |
| Results | |
| dashboard | Yes grafana dashboard http://onapci.org/grafana | Yes | Yes Based on xtesting, all results pushed to a mongo DB (same than for infrastructure tests) Several web front end (grafana, Kibana, static web pages) used as dahsboards | |
+ VTP would use dovetail to perform these test cases
* VTP uses ONAP commands as done here https://github.com/onap/integration/tree/master/test/hpa_automation