Vendor Landscape Methodology
Info-Tech Research Group’s Vendor Landscapes review a particular IT market space. We evaluate the strengths and abilities of the products available in that space as well as the vendors of those products. Evaluations weigh selected vendors and their products (collectively “solutions”) on the following eight criteria to determine overall standing:
|Product Evaluation Criteria||Vendor Evaluation Criteria|
Evaluated solutions are plotted on a standard two-by-two matrix:
- Champions: Product and vendor scores are above the evaluation group’s average.
- Innovators: Product score is above the average but vendor score is below.
- Market Pillars: Vendor score is above the average but product score is below.
- Emerging Players: Product score and vendor score are below the average.
Info-Tech determines which solutions will be included in a particular Vendor Landscape based on a combination of analyst experience and client demand. Additionally, analysts may determine what specific capabilities are required of the evaluated solutions and include only those solutions that meet the defined requirements.
Info-Tech Research Group gathers information from a variety of sources:
- From vendors via a survey, briefing and product demonstration, and pricing submission
- From end users via survey and interview
- From third parties via secondary research
Refusal by a vendor to participate does not eliminate a vendor from inclusion in the evaluation.
Scoring occurs once all information gathering is complete, for consistency. Scores are transposed into Info-Tech’s Vendor Landscape Shortlisting Tool, which automatically determines Vendor Landscape positioning, Criteria / Harvey Ball Score, and Value Index.
Info-Tech presents the information it collects in a number of different ways, each designed to give readers a different insight into the analysis performed.
Info-Tech’s Vendor Landscape is a two-by-two matrix that plots solutions based on the combination of Product score and Vendor score. Placement is relative based on weighted aggregate criteria scores.
Criteria Scores (Harvey Balls)
Info-Tech’s Criteria Scores are visual representations of the absolute score assigned to each individual criterion as well as of the calculated overall Vendor and Product scores. Harvey Balls are used as the visual representation.
VFeature Ranks (Stop Lights)
Info-Tech’s Feature Ranks are visual representations of the presence/availability of individual features that collectively comprise the Features criterion. Stop Lights are used as the visual representation.
Info-Tech’s Value Index is an indexed ranking of solution value per dollar as determined by the raw scores assigned to each criterion.
Where pricing is not provided by the vendor and public sources of this information cannot be found, an Affordability Raw score of zero is assigned. Since multiplication by zero results in a product of zero, those solutions for which pricing cannot be determined receive a Value score of zero.
Info-Tech’s Price Evaluation is a tiered representation of the three-year Total Cost of Ownership (TCO) of a proposed solution. The solution TCO is calculated and then represented as belonging to one of ten pricing tiers.
Where pricing is not provided, Info-Tech makes use of publicly available sources of information to determine a price. Because these sources are not official price lists, the possibility exists they may be inaccurate or outdated, and so the source of the pricing information is provided.
Info-Tech’s Price Evaluations are based on pre-defined pricing scenarios to ensure a close “apples-to-apples” comparison between evaluated solutions.
Info-Tech’s Scenarios highlight specific use cases for the evaluated solution to provide a basis for comparison that is as complete as possible for end user clients. Scenarios are designed to reflect tiered capability in a particular set of circumstances. Where possible, Scenarios are designed to be mutually exclusive and collectively exhaustive, or at least hierarchical, so that the tiers within the Scenario represent a progressively greater or broader capability.
At the conclusion of all analyses, Info-Tech presents awards to exceptional solutions in three distinct categories.
Awards categories are as follows:
- Champion Awards are presented to those solutions, and only those solutions, that land in the Champion zone of the Info-Tech Research Group Vendor Landscape.
- Trend Setter Awards are presented to those solutions that are deemed to include the most original product/service, or the most original capability of a product/service.
- Best Overall Value Awards are presented to those solutions, and only those solutions, that are ranked highest on the Info-Tech Research Group Value Index.
Info-Tech takes the factual accuracy of its Vendor Landscapes seriously. To ensure the utmost accuracy, all vendors of evaluated solutions are invited to participate in a process of Fact Check.
Once the research project is complete and the materials are deemed to be in a publication-ready state, excerpts specific to each vendor’s solution are provided to the vendor.
Info-Tech provides a one-week window for each vendor to provide written feedback. Feedback must be corroborated, and where it is, feedback that addresses factual errors or omissions is adopted fully while feedback that addresses opinions is taken under consideration.
Publication of materials is scheduled to occur within the twelve weeks immediately following the completion of the research project, but does not occur until the Fact Check process has come to conclusion and under no circumstances are pre-publication copies of any materials made available to any client.
VL Research Agenda
The market-spaces we will cover and the current vendors list are available in this PDF.
The complete Methodology is available in this PowerPoint Deck.
Questions about our Vendor Landscape Methodology?
Request a Briefing