One portal should support multiple programs, that can be searched / filtered by program/badge type on the public listing page.
The public lists should searchable, and allow filtering by the program type, company, and other columns displayed on the main page.
High-level use cases
Support upload, validation, display, sharing, and manage test results and application by "user"
Support a review workflow of test results by "reviewers"
Publicly list companies and products which have obtained a badge in a "marketplace", as marked by "admin"
Test Result Management
authenticated users (role "user") must be able to
upload test results
edit meta data (application) of a test result set (product name, etc.)
view, delete, and edit only their own test results
change status of a test result between "private" and "for review"
review management
authenticated reviewers (role "reviewer") must be able to:
access only to test results set to state "for review" (not all uploaded results)
cast a vote (-1, 0, 1) on instance of test results submitted to review
Add comment along with there vote (i.e. why they voted -1, etc.)
OVP release management
Management of releases of OVP (create new, edit, delete) must be runtime operations, i.e., not requiring new versions of the portal
a OVP release comprises
a unique identifier (e.g. OVP020.09)
links to documentation
a list of test cases for each program type that are mandatory or optional
This list is used to validate if a set of submitted results meets the requirements for the OVP release.
portal lifecycle management
all management operations on test results, market place entries, users, and new releases of OVP must be runtime operations not requiring new builds of the web portal
separation of LCM of the portal instance (responsibility of LF IT) and content (responsibility of OVP admins)
Public List Management
"admins" (user role) must be able to manage entries of the marketplace (create, edit, delete)
all entries of the marketplace must be stored in persistent storage
entries must include support to display a company logo (provided by the user submitting the application)
market place data items per entry: see current fields + <add more if needed, Brandon?>
A full list of fields for the existing NFVI and VNF programs will be provided by the LFN CVC
LFN CVC will provide guidance on which fields will display on the top level list (main page) or only in a detailed listing (linked to from the main page)
User Management
Users log in through a Linux Foundation Open ID
A user logging in for the first time is automatically assigned the "user" role.
User Roles
"user"
Can upload and manage test results
Can only see own test results
Can create an application to submit their results to review
"reviewer"
can see all test results marked as "for review" by its "user" (the user is the owner of the results / application)
"admin"
can manage assigned user roles
can manage (create, update, delete) entries to the marketplace
A portal user can have multiple assigned roles, i.e. Jo can be assigned the roles of "admin" and "user"
Results Format
Should be as flexible as possible.
Results are uploaded as a zip or tar.gz file, other formats will be rejected
Must include a "test result summary" in the archive file root
The "test results summary" will include:
Version of the tool used (i.e. what version of functest was running)
Date & time of the test run
validation and display of test results (see also terminology below)
the web portal must validate uploaded test results by comparing the "test result summary" to a "test result guideline"
"test result guideline": source of truth
list of all test cases which are part of a given OVP release
use case: detect if test cases are missing from uploaded test results
the expected result for passing each test case (functional tests: "pass", non-functional: "value")
stored in web portal only
"test result summary"
part of the "test result package" generated by test tool
json formatted
should include in addition to today (AP on test tooling team)
OVP release ID (e.g. 2020.10)
OVP program type (e.g. NVFI, VNF, ...)
example of a "test result package" currently generated by test tooling:
