Vendor Landscape Evaluations

Research Memberships

Info-Tech Research Group's Vendor Landscape market evaluations are a part of a larger program of vendor evaluations which includes Solution Sets that provide both Vendor Landscapes and broader Selection Advice.

Comparing Vendors

From the domain experience of our analysts as well as through consultation with our clients, a vendor/product shortlist is established. Product briefings are requested from each of these vendors, asking for information on the company, products, technology, customers, partners, sales models and pricing.

Our analysts then score each vendor and product across a variety of categories, on a scale of 0-10 points.The raw scores for each vendor are then normalized to the other vendors' scores to provide a sufficient degree of separation for a meaningful comparison.

These scores are then weighted according to weighting factors that our analysts believe represent the weight that an average client should apply to each criteria. The weighted scores are then averaged for each of two high level categories: vendor score and product score. A plot of these two resulting scores is generated to place vendors in one of four categories: Champion, Innovator, Market Pillar, and Emerging Player.

Vendor Landscape diagram sample

The Vendor Landscape Diagram Explained

• Champions receive high scores for most evaluation criteria and offer excellent value. They have a strong market presence and are usually the trend setters for the industry.
• Innovators have demonstrated innovative product strengths that act as their competitive advantage in appealing to niche segments of the market.
• Emerging Players are newer vendors who are starting to gain a foothold in the marketplace. They balance product and vendor attributes, though score lower relative to market Champions.
• Market Pillars strike a strong balance between product and vendor attributes. They have the potential to become future industry leaders if they address the missing links in their offerings.

Category Comparisons

For a more granular category by category comparison, analysts convert the individual scores (absolute, non-normalized) for each vendor/product in each evaluated category to a scale of zero to four whereby exceptional performance receives a score of four and poor performance receives a score of zero. These scores are represented with "Harvey Balls", ranging from an open circle for a score of zero to a filled in circle for a score of four. Harvey Ball scores are indicative of absolute performance by category but are not an exact correlation to overall performance.

Sample category comparison using "Harvey Balls".

Individual scorecards are then sent to the vendors for factual review, and to ensure no information is under embargo.

How we use your feedback:

• We will make corrections where factual errors exist (e.g. pricing, features, technical specifications).
• We will consider suggestions concerning benefits, functional quality, value, etc; however, these suggestions must be validated by feedback from our customers.
• We do not accept changes that are not corroborated by actual client experience or wording changes that are purely part of a vendor's market messaging or positioning.

Any resulting changes to final scores are then made as needed, before publishing the results to Info-Tech clients. Vendor Landscapes are refreshed every 12 to 24 months, depending upon the dynamics of each individual market.

Value Index Ranking Methodology

Info-Tech Research Group's Value Index is part of a larger program of vendor evaluations which includes Solution Sets that provide both Vendor Landscapes and broader Selection Advice.

Sample value index: composit performace score

How It Works

The Value Index is an indexed ranking of value per dollar as determined by the raw scores given to each vendor by analysts. To perform the calculation, Affordability is removed from the Product score and the entire Product category is reweightedto represent the same proportions. The Product and Vendor scores are then summed, and multiplied by the Affordability raw score to come up with Value Score. Vendors are then indexed to the highest performing vendor by dividing their score into that of the highest scorer, resulting in an indexed ranking with a top score of 100 assigned to the leading vendor.

The Value Index calculation is then repeated on the raw score of each category against Affordability, creating a series of indexes for Features, Usability, Viability, Strategy and Support, with each being indexed against the highest score in that category. The results for each vendor are displayed in tandem with the average score in each category to provide an idea of over and under performance.

The Value Index, where applicable, is refreshed every 12 to 24 months, depending upon the dynamics of each individual market.

Questions about our Vendor Landscape Methodology?
Ask Questions Here