Use is not the same as adoption
After a new system goes live, the first report is usually a count of accounts created or people trained. Those figures are easy to produce and they show that the rollout reached people. They do not show whether the system is being used in the way it was designed, or whether it is helping anyone.
Two situations illustrate the gap. In the first, everyone logs in daily because they must, but they copy data into the old spreadsheet and do the real work there. In the second, fewer people log in, but those who do complete the whole process in the system and no longer need side records. A login count ranks the first as better. Adoption measurement is about telling these cases apart, so that decisions about further support rest on what is actually happening.
Nominal compliance versus real use
It helps to think of adoption in layers.
- Access: people have accounts and can open the system.
- Activity: people perform some actions in it.
- Completeness: the full process is carried out in the system, with accurate and timely data.
- Dependence: other people and processes rely on the output, and old methods have stopped.
- Outcome: the purpose behind the change, such as shorter handling time or fewer errors, is being met.
Most early reporting stops at the first two layers. A fair assessment looks further along, while accepting that outcome data is harder to attribute to a single cause because other things change at the same time.
Signals worth tracking
No single signal is reliable. Combining a few from different layers gives a more balanced view.
Usage patterns. Frequency and breadth of use by team or role. Look for the variation, not just the average: a healthy overall figure can hide one department that has quietly opted out.
Completion and quality. The share of transactions or requests that go through the system from start to finish, and how often records are missing required information or need correcting. Poor data quality in the first weeks may point to training gaps or confusing screens rather than reluctance.
Workarounds. Parallel spreadsheets, emailed requests that should have been tickets, paper forms, and informal requests to a colleague who then enters the data. Workarounds are among the clearest signs of nominal compliance. They are also information: they often show where the system does not fit the work.
Support demand. Volume and type of questions and tickets. A spike in “how do I” queries suggests training needs; persistent complaints about the same step suggest a design issue. Service desk metrics from the support team can help, though ticket counts alone say little without context.
Time and effort. Whether tasks take longer or shorter than before. This requires a performance baseline taken before the change; otherwise comparison depends on memory.
People’s own accounts. Short surveys, interviews and conversations with team leads. They capture reasons that numbers cannot, such as “the approval screen is slower than the old form”.
Downstream effects. Whether reports, handoffs and decisions now draw on the system, or whether people still ask for the figures to be assembled by hand.
Setting targets without distorting behaviour
Targets focus attention, but they also change behaviour. If managers are judged on how many staff log in weekly, logging in will increase, and not necessarily useful work. If teams are measured on records created, some will create more records than they need. These target side effects are well known in any measurement program, and adoption is no exception.
A few precautions reduce the risk:
- Prefer a small set of measures that cover different layers over a single headline number.
- Pair volume measures with quality measures, such as records created and records needing correction.
- Be clear about what a measure means, through written metric definitions, so teams do not argue about whether a figure is comparable.
- Treat early numbers as diagnostic, not as grounds for ranking individuals.
Measuring fairly
The aim of adoption measurement is to improve the change, not to find people to blame. Several practices help keep it fair.
Compare like with like. Teams with different workloads, locations or starting systems will not adopt at the same pace. Comparing a small, simple team with a complex one can mislead.
Allow for a learning curve. Productivity often dips and then recovers as people become familiar with a tool. Judging after one week may overstate the problem; judging only after six months may leave real difficulties unnoticed. Agreeing in advance when to check, and what pattern is reasonable, helps.
Look at the system as well as the user. If a large group is not using a feature, the cause may be the design, the training, or a policy that conflicts with it. Treat low adoption as a question about the whole change.
Separate monitoring of work from monitoring of people. Aggregated, team-level data is usually enough for adoption purposes. Detailed tracking of individuals may raise privacy and trust concerns and, depending on the setting, legal or employment considerations, so it is worth checking with HR or legal advisers before collecting it.
Be open about what is measured. Explaining the purpose and sharing the results with the teams involved builds trust, and invites explanations for odd figures.
Turning findings into action
Measurement is only useful if it leads somewhere. Common responses include:
- additional or different training, perhaps through system training approaches better suited to the group concerned;
- changes to the configuration or process when many people share the same difficulty;
- targeted help for teams that appear to be struggling;
- retiring old methods where they are only being retained out of habit;
- adjusting the plan if the original expectations were unrealistic.
If adoption stays low and the reasons are not clear, a structured diagnosis, as covered in stalled change diagnosis, can separate legitimate objections from habit. Sometimes the finding is that the system is a poor fit, and that is useful information too.
A simple review rhythm
A modest schedule is often enough: a first check within the first few weeks to catch obvious problems, another at around the point when people should be comfortable, and a later review to see whether benefits are showing up. A compact operations dashboard can bring the main signals together, as long as it displays the context alongside the numbers. At each review, ask what the figures suggest, what they might be hiding, and what the people involved would say about them. Numbers raise questions; conversations usually supply the answers.