Community Forum – How do you measure the quality of an idea?

Our organization is looking to understand how the front end of innovation is motivated and organized to feed the R&D pipeline.  When projects have large uncertainties, what are some key attributes that influence early investment decisions? 

1. What criteria or metrics does your organization use to judge the quality an idea?

• Marketing Criteria (TAM, NPV) based on ‘t-shirt’ sizing opportunity;  ROII (cutting the tail);  Customer Engagement (VOC, Needs, Concept feedback under NDA);  Customer collaboration
• Strategic alignment; ability to win; market situation (new; growing; mature; etc.)
• Technical evaluation, “Is it worth it?”, “Can we win?”
• Technical feasibility, manufacturing feasibility, cost, market/strategic importance
• MUST HAVE a clear alignment with the corporate vision and business strategies.
• Idea has to be relevant to technology we practice or to a clear market opportunity of interest.
• Review and commitments by global functional leaders
• Magnitude, connection to market need, and confidence of success.
• Risk adjusted value metric
• Benchmarking with customer
• Alignment with strategy, risk uncertainty, ROI
• Fourth Generation Innovation including 4G R&D
• Originality and feasibility
• Does it meet military medical mission and is there a champion beyond the inventor.
• NPV, Sales Management consensus
• Revenue potential at maturity; IP protection potential; business buy-in.
• Desirability, feasibility and viability.  Focus first on the potential strength of value proposition and customer connection, then do we have the technology to provide the solution, then is there $ opportunity.
• None

2. Are any key attributes validated with experts before funding a new idea? If yes, what level of validation is performed?

• Needs Unique Value Proposition.  
• It depends on the type of R&D project. Technology Development efforts (Horizon 2) are validated by cross-functional technology roadmapping sessions that rely heavily on internal experts, market reports, and (sometimes) external consultants/coaches. For large B2B Product Development efforts, we validate with market reports, direct market research, and feedback from beta-sampling partners.
• Literature / Patent survey, expert interviews, strategic fit discussed with responsible executive manager (business unit CTO)
• High level cost modelling, probability of technical/commercial success.
• An early techno-economic feasibility to kill it fast; If the idea survives, develop what its commercialization plan may look like.
• We have a panel of fellows that help with validating ideas and a reasonable next step is to fund a 3-6 month exploratory project paid with innovation funds.
• Risk assessments, project plans, resource plans are considered primarily – attempt to answer “what will be the impact of the project (e.g. revenue expected) and who is interested (e.g. what brand/region will launch the technology)”
• Customer or market sponsorship validates value of unmet need and sets success criteria.
• We increasingly use experts outside of the company to validate our value proposition and other business assumptions.
• Interviews
• Yes, a notable number of preferably blinded interviews.
• Experts frequently block radical innovation – Subset of 4G is Lean Startup -both 4G and LS do customer testing.
• Yes – through Chief Scientist/Engineer and Technical Director review
• Yes, very formal process of evaluation for price, efficacy, fieldability, sustainability
• Market size Sales price Margins at Maturity
• Yes, Key business/technology leader buy-in.
• Yes, Lots of customer/market interviews to validate the need.  Depending on the technology differences, will also include outside technical experts.
• PowerPoint pitch

3.  What criteria or metrics are used to judge the quality of the opportunity space?

• We work within defined core-markets
• Fit with our core technical strengths; alignment to one or more key strategic initiatives or target markets; ability to leverage the idea/project across multiple applications
• Diversity, inclusion of “radical” ideas, iterative process of refining ideas and internal pitches.
• Market interest, strategic business fit.
• Identify assumptions that need validation; confirm what we know and what we don’t know.
• We don’t have specific metrics to apply, but the originator would need to estimate the market opportunity in a form of potential volume and/or $ annual sales.
• We decide by committee, specific metrics are not clear – typically driven by regional/brand/business interest and commitment.
• Literature precedent or proof of concept demonstration for the chemistry and technical effect, prototyping.
• Market attractiveness, differentiation, strength of value proposition, business case
• Review of a business model canvas along with other input.
• Quality has several factors- one the scalable size of market another is timing another us size of investment another is profit margin another is competitive threat size.
• Business Capabilities, Competition, Potential ROI
• Civilian and/or military opportunities are both evaluated.
• Overall Technical, Market and Manufacturing fit
• Manager judgement
• Not quite following the question- we don’t have metrics around a set of ideas. The metrics are on the idea itself.
• Market size

Do you have a question for the IRI Community Forum? Email Us Today!