• What are we doing?
    • Developing metrics for measuring quality, value, and realization of benefits of badging to RAx standards
    • Below Table: proposed Key Metrics for 2020.
      • Presented in Prague, CZ, January 13 - 16, 2020, and ongoing work in progress.
    • Designing and building data analytics tools as needed to gather and support data analysis
      • LF Developer Analytics Dashboard
  • Why are we doing it?
    • Establish platform to gather feedback on approach, metrics, and data.
    • Provide data driven insights to support CNTT mission and goals.


#AreaFunctionSegmentsTargeted BenefitsMetricPriority
(Urgency)		Data Source(s)Benchmark2020 TargetFutureComment
1GovernanceEngagementAllIncrease engagement of SP community membersCode/content contribution rates - by  company1GitHub, Groups.io, Confluencewipwipwip
2GovernanceEngagementAllIncrease engagement of SP community membersCode/content contribution rates - by individual1GitHub, Groups.io, Confluencewipwipwip
3GovernanceAdoptionAllImprove Onboarding and Implementation

TBD: How to quantify TC metrics? (trial, cookbook, etc.) 

  • Functional
  • Non-functional
    • Strawman for Pre-Launch Trials
      • Number of days to install RI-1
      • Number of hours to run RC-1 
  • What are the Pass/Fail criteria for each RI? RC?
  • What is a ‘good’ result? I.e. what does it mean and why should I care?
1wipwipwipwip

We need to develop benchmarks for pre-launch trials, e.g. is 50%, 70%, 85% pass rate 'good' for TCs pass/fail of expected results for RI?

https://github.com/cntt-n/CNTT/issues/1097

4GovernanceRecruitmentService ProvidersIncrease acquisition of community membersNew member acquisition rate - by company2wipwipwipwip
5GovernanceRecruitmentVendor PartnersIncrease acquisition of community membersNew member acquisition rate - by company2wipwipwipwip
6GovernanceEngagementLabsIncrease engagement of community members# of labs with CNTT verification capability2wipwipwipwip
7GovernanceAdoptionService ProvidersIncreased adoption by SPsrate of adoption of RAx - by company2wipwipwipwip
8GovernanceAdoptionService ProvidersIncreased adoption by SPs# of RFPs published required CNTT badged VNF2wipwipwipwip
9Technology ConformanceService ProvidersReduce costsSee TCO Analysis2wipwipwipwip
10GovernanceAdoptionVendor PartnersIncrease adoption by VPsrate of badges issued - by company3wipwipwipwip
11Technology ConformanceService ProvidersAccelerate time to deployment for new network services# of weeks to deploy VNF product into lab (or deployment)3wipwipwipwip
12Technology ConformanceService ProvidersImprove interoperability and software quality(TBD: criteria: technical, evolutionary, configuration, security, operations, performance)3wipwipwipwip
13Technology ConformanceService ProvidersReduce in-house testing effort# of weeks to test VNF product in lab to deploy (time to productivity) 3wipwipwipwip
14Technology ConformanceVendorsImprove time to revenue for new product offerings# of weeks to develop new VNF3wipwipwipwip
15Technology ConformanceVendorsAchieve greater alignment with service provider customer requirements% of VNFs badged - by company3wipwipwipwip
16Technology ConformanceVendorsDemonstrate product quality through open ecosystem testing# of badges issued - by company3wipwipwipwip
  • No labels