What a scoring matrix does
A weighted scoring matrix is a table. The rows are the criteria that matter, such as functional fit, integration, usability, support and cost. The columns are the options. Each criterion has a weight showing how much it matters, each option receives a score against each criterion, and a total is calculated.
Its main value is not the final number. It is that it forces a group to say, in writing and before looking at products, what matters and how much. It also makes disagreements specific: instead of “I prefer option B”, someone says “I scored its integration lower because the connector only covers one of our two systems”.
Building one
1. Pick criteria from your requirements. Take them from the written requirements rather than inventing new ones halfway through. Limit yourself to perhaps six to ten criteria; more than that tends to dilute the important ones. Group related items, and avoid listing the same quality twice under different names, which silently doubles its weight.
2. Separate must-haves from scored criteria. Mandatory requirements work best as a pass or fail gate applied first. An option that fails a must-have should not be rescued by high scores elsewhere. Scoring is for comparing options that already clear the gate.
3. Set weights. Assign each criterion a share of the total, for example out of 100. Agree weights as a group, before scoring, and write down the reasoning. Doing this before seeing any scores reduces the temptation to adjust weights until a favourite wins.
4. Define the scale. A scale of 1 to 5 is common. Describe what each score means for each criterion, for example “5 means the requirement is met fully in the trial, 3 means it is met with a workaround, 1 means it is not met”. Without definitions, one person’s 4 is another’s 2.
5. Score from evidence. Base scores on trial results, documents, reference calls and supplier answers, not on impressions from a presentation. Have evaluators score independently first and then compare, which exposes differences in interpretation.
6. Calculate and review. Multiply each score by its weight, add up the results, and divide by 100 (the total weight) to get a weighted average on the original scale.
A worked illustration
The figures below are invented purely to show the arithmetic. Two options, X and Y, are scored from 1 to 5 against five criteria.
| Criterion | Weight | Option X score | Option Y score |
|---|---|---|---|
| Fit to requirements | 35 | 4 | 5 |
| Integration | 20 | 3 | 4 |
| Usability | 20 | 5 | 3 |
| Support | 15 | 3 | 4 |
| Cost | 10 | 3 | 2 |
Option X: (35 x 4) + (20 x 3) + (20 x 5) + (15 x 3) + (10 x 3) = 140 + 60 + 100 + 45 + 30 = 375. Dividing by 100 gives 3.75.
Option Y: (35 x 5) + (20 x 4) + (20 x 3) + (15 x 4) + (10 x 2) = 175 + 80 + 60 + 60 + 20 = 395, which gives 3.95.
On these weights, Y leads by 0.2 points. Notice how small the gap is.
Test how sensitive the result is
Now change the weights modestly. Suppose the group decides that usability deserves more attention: fit becomes 25, integration stays 20, usability becomes 30, support stays 15 and cost stays 10 (the total is still 100).
Option X: 100 + 60 + 150 + 45 + 30 = 385, or 3.85.
Option Y: 125 + 80 + 90 + 60 + 20 = 375, or 3.75.
The ranking has reversed, simply by shifting ten points of weight. This kind of check is valuable. If a modest, defensible change in weights flips the winner, the matrix is telling you the options are close, and that the deciding factor should be discussed openly rather than hidden in arithmetic.
Limits to keep in mind
- False precision. Scores are judgments. A total of 3.95 against 3.75 looks exact but rests on subjective inputs, so small gaps should not be treated as meaningful.
- Weights carry the decision. Whoever sets the weights has great influence, and they can be tuned, knowingly or not, to favour a preferred option.
- Averaging hides deal-breakers. A very low score on one critical item can be offset by high scores elsewhere. Use gates for the critical items and consider reviewing any criterion scoring 1 separately.
- Not everything fits a number. Vendor stability, culture fit and long-term strategy resist scoring, and are better discussed in words.
- Cost needs care. Scoring price as one criterion among many can understate its importance or hide it. Many teams score quality criteria first and then compare the results against total cost of ownership separately.
- Double counting. Overlapping criteria inflate the weight of one concern.
Using it well
Treat the matrix as one input among several, alongside trial results from a software pilot, reference checks and answers gathered in vendor due diligence. Share the sheet with the decision owner and keep it in the decision record with the reasoning behind the weights. If the matrix and your instinct disagree, that is useful: examine why, and which of the two is missing something. A tool for structuring judgment is helpful; replacing judgment with a spreadsheet is where the trouble begins.