Page History

...

• General requirements
  • the portal must at least provide the same functionality as today's portals (see https://nfvi-verified.lfnetworking.org/#/):
  • high-level use cases "marketplace": list companies and products which have obtained a badge
    1. upload, validate, display and manage test results by "uploaders"
    2. review test results by "reviewers"
    3. list companies and products which have obtained a badge in a "marketplace"

• Integration of different OVP programsOVP program integration
  • the web portal must integrate different types of OVP programs (e.g. NVFI, VNF, ...)
  • today, the VNF and NFVI program use separate web portals basically providing the same functionality, only differing in the validation of test results.

• user roles
  • "uploader"
    • can upload and manage test results
    • can only see own test results
  • "reviewer"
    • can see all test results marked as "for review" by its "uploader"
  • "admin"
    • can manage user roles
    • can manage (create, update, delete) entries to the marketplace

• test result management
  • authenticated users (role "uploader") must be able to
    • upload test results
    • edit meta data of a test result set (product name, etc.)
    • view and edit only their own test results
    • change status of a test result between "private" and "for review"

• review management
  • authenticated reviewers (user role) must be able to
    • access all test results set to state "for review"
    • cast a vote (-1, 0, 1) on every instance of test results

• OVP release management
  • management of releases of OVP (create new, edit, delete) must be runtime operations, i.e., not requiring new versions of the portal
  • a OVP release comprises
    • documentation
    • a unique test result schema for test result validation and display

• validation and display of test results (see also terminology below)
  • the web portal must validate that an uploaded set of test results conforms to a given format (schema)
  • validation of test results must follow a schema (manageable as part of the release management)
  • the schema shall include
  • by comparing the "test result summary" to a "test result guideline"list of all tests per OVP release: "source of truth" - does an uploaded test result package contain results for all tests of an OVP release
    • "test result guideline": source of truth
      • list of all test cases which are part of a given OVP release
        • use case: detect if test cases are missing from uploaded test results
      • the expected result for passing each test case (functional tests: "pass", non-functional: "value")
      • stored in web portal only
    • "test result summary"
      • part of the "test result package" generated by test tool
      • json formatted
      • should include in addition to today (AP on test tooling team)
    • • • OVP release ID (e.g. 2020.10)
        • OVP program type (e.g. NVFI, VNF, ...)
  review management
  • • • example of a "test result package" currently generated by test tooling:
        • http://artifacts.opnfv.org/functest/DUXGBAW88H7Q.zip
  • optional requirements, requires close collaboration with and input from test tooling team
    • define a schema for formal validation of test result summary
    • define a schema for formal validation of test result guide
  • reviewers (user role) must be able to
  • access all test results set to state "for review"
  • cast a vote (-1, 0, 1) on every instance of test results

• portal lifecycle management
  • all management operations on test results, market place entries, users, and new releases of OVP must be runtime operations not requiring new builds of the web portal
  • separation of LCM of the portal instance (responsibility of LF IT) and content (responsibility of OVP admins)

...

• user management
  • allow for managing users and roles based on LF IDs
  • roles: reviewer, uploader, admin

• Terminology
  • "test result package": archive containing "test result summary" file and individual logs
  • "test result summary": json formatted file containing a summary of all test cases / one run of the compliance test tool
  • "test result guideline": json formatted file containing all tests which are part of an OVP release + expected result for passing a test

...

Requirements (as noted during the call on 18 May 2020:

...