Once you decide to measure, what you measure with matters

You have decided to measure at the start, alongside what learners tell you. Anyone, or an AI, can build timed tasks. Tasks refined over years, a norm sample, scoring rules and evidence that it measures what it claims are what turn them into an assessment. What makes an assessment worth trusting is the evidence behind its scores, and the five things below are what that evidence must show.

It became a battle. The specific gap the EPAO pointed to was processing speed. There were no numbers, no testing. They couldn’t see how we could validate that a learner needed extra time.

Zoe Fox, Head of Skills and Inclusion, Aspiration Training

End-point assessment organisations (EPAOs) were rejecting the self-report assessment Aspiration Training used as evidence. They replaced it with a validated cognitive assessment at enrolment, and their May 2026 Ofsted inspection judged them at the expected standard across every area.

Read how Aspiration Training started measuring

In my experience, CMI readily recognise the Cognassist assessment report as a valid form of evidence when applying for reasonable adjustments.

Philippa Gamble, Additional Learning Support Lead, Fuel Learning

Fuel Learning’s additional learning support lead, on applying to CMI, an awarding body, for learners’ reasonable adjustments.

Five things that make a score worth trusting, in plain words

Each one starts with something you already know from teaching or everyday life, then gives the name the science uses, so you can recognise it in a report or a sales conversation.

A score compared with whom?

The growth chart in a baby’s red book: a height means nothing until you compare it with children the same age. A score means nothing on its own; it means something when it is compared with people like the learner. That comparison group is the norm sample, and its size and make-up decide whether the comparison is fair. So ask who the scores are compared with, and how many.

A score compared with whom?

What does a score of 84 actually mean?

Most people score between 85 and 115 for their age: that is the typical range, with 100 in the middle. A score of 84 is the first score below the typical range. The line at 84 is drawn where the exam boards’ own guidance draws it, not where we chose to put it.

  • The term: these are standard scores, with 100 the average for people the same age. A score of 84 is at the 14th percentile, so 14 in every 100 people that age score lower.
  • What it is not: a score below the typical range says the learner found these tasks harder than most people their age. It is a reason for a conversation, not a diagnosis.
An illustration of a score scale from 70 to 130. The span from 85 to 115 is shaded and labelled typical range, with 100 at its centre. A marker at 84 sits just outside the band's left edge.
Illustration, not Cognassist data

What does a score of 84 actually mean?

If the learner took it again, would they get a similar score?

You couldn’t build anything with a tape measure that gives a different length each time. A score needs evidence it would come out similar again. Reliability is how consistent a score is: a learner who took the same assessment again a few weeks later should get much the same result. The more reliable a score, the smaller its margin of error, and the more safely you can act on it.

If the learner took it again, would they get a similar score?

Why does every learner take it the same way?

Markers drift as they tire: by the end of a long morning, the same standard of work gets lower marks. Standardisation means every learner gets the same instructions, the same practice, the same tasks and timing, and the same scoring, so a difference in score comes from the learner, not from who gave the assessment or who marked it. So ask whether every learner takes it the same way.

Why does every learner take it the same way?

Does it measure what it says it measures?

A spelling test that is really a handwriting-speed test: a learner who spells every word right but writes slowly scores low. Validity is the evidence that an assessment measures what it claims to, for learners like yours. It is shown three ways: the tasks group together the way the theory says they should, the right learners score differently, and the scores link to outcomes that matter.

Does it measure what it says it measures?

How do you know the learner actually tried?

Think of the learner who races through an online induction module, ticking boxes to get to the end. Learners take an online assessment on their own, so the same can happen. Performance validity checks, built into each task, catch a learner who is clicking through, not engaging or interrupted. A score that fails its check is set aside rather than trusted. You may know them as engagement checks.

How do you know the learner actually tried?

Six questions to ask any assessment before its price

Four of them test the ideas above; the last two ask who checks the science and how a score is used. These questions need evidence to answer, so ask them of any assessment, ours included. Our answers follow.

Six questions to ask any cognitive assessment

  1. Is it normed on learners like yours, and how many? Tests who a score is compared with. A comparison is only fair when the group is big enough and close enough to your learners. A good answer: A named group of people like your learners, how many there are and scores adjusted for age.
  2. How reliable are the scores used for decisions? Tests whether a score would come out much the same if the learner took it again, so you can act on it. A good answer: Published figures for the scores used in support decisions, checked by assessing the same people again, weeks apart.
  3. Does it measure what it says, and predict a real outcome? Tests whether a score means what it claims, for learners like yours, and links to something that matters on the course. A good answer: Evidence shown all three ways, with its limits stated plainly.
  4. Does it check the learner was engaged? Tests whether the learner really tried. A score from a learner clicking through tells you nothing about them. A good answer: Checks built into the tasks, and a score that fails one set aside, not passed on.
  5. Who oversees it independently? Tests what no one can simply say about themselves: whether people outside the company check the science. A good answer: Named people or bodies outside the company, with the expertise to judge it.
  6. Is it ever the only basis for a decision? Tests how the score is used. An assessment informs a decision about support; it should never make that decision alone. A good answer: No. It starts a conversation, and staff decide with what they know of the learner.

How our cognitive assessment answers them

The same questions, put to ours and answered from the published evidence. Where we’d be careful comes straight after.

Question one

Each learner is compared with 26,865 UK apprentices, by age

The comparison group, the norm sample, is 26,865 UK apprentices of working age, with an average age of 29.4 and 60% female, drawn from 42,484 learners assessed through training providers. Every score is adjusted for age, so a learner is read against what is typical at their age.

  • Its size: more than 10 times the normative sample of the Wechsler Adult Intelligence Scale, fifth edition (WAIS-5), the most widely used one-to-one adult cognitive test, which was normed on 2,020 people.
  • Kept current: the group keeps growing through routine use, with checks before anyone new is added.

A score compared with whom?

Question two

The scores used for decisions meet a good-to-excellent standard

The assessment produces three index scores, the scores used to judge whether support may be needed. All three are reliable to a good-to-excellent standard. The retest study had 112 people taking the assessment twice, at least 30 days apart.

  • Language & Numbers and Visual & Fluid lead, with internal consistency of .91 and .93, and test-retest of .87 for both: above the British Psychological Society’s (BPS) .80 benchmark for high-stakes decisions.
  • The Memory index is lower and sits just below that benchmark.
A bar chart of reliability for the three index scores, internal consistency and test-retest, against a line at .80 marked as the British Psychological Society's high-stakes benchmark: Language & Numbers .91 and .87, Visual & Fluid .93 and .87, Memory .78 and .68.
Source: Cognassist Science, Validation White Paper Series, Paper 1 (2026)

Read the validation papers

If the learner took it again, would they get a similar score?

Question three

Independently validated, with evidence all three ways

The cognitive assessment has been independently validated as accurate and robust, and reviewed by the Professional Association of Teachers of Students with Specific Learning Difficulties (PATOSS) against the Joint Council for Qualifications’ (JCQ) scientific-validity requirements. The evidence covers all three ways validity is shown.

  • The tasks group as theory says: three groups that match established models of how people think, and those groups are the three index scores.
  • The right learners differ: the cognitive differences associated with dyslexia show on reading decoding. The assessment stays specific where no difference is expected, and doesn’t flag ADHD or autistic learners as having a learning-difference profile.
  • Scores link to outcomes: among 588 apprentices at three providers, the cognitive profile predicted withdrawal better than a diagnosis label; those flagged in two areas withdrew at 86%, against 77% with no flag.

The second and third are doctoral research at Newcastle University, independently supervised and being prepared for publication.

Does it measure what it says it measures?

Question four

The same way for every learner, with an engagement check in every domain

Every learner gets the same plain-language instructions and answers a practice question before anything counts. Scoring is fully automated and identical for every learner, so no marker can drift.

  • Each of the nine domains carries its own engagement check. If a check fails, that score is set aside, not passed on as if it were real.
  • What it means for you: a score that reaches you was taken the same way as everyone else’s and passed its check, so it is a score about the learner.
The practice question before the Literacy task

See what taking the assessment looks like

How do you know the learner actually tried?

Questions five and six

Overseen by named experts, and never the only basis for a decision

The assessment was developed with an Independent Science Advisory Board of four named external experts, who actively oversaw development: every task was chosen and developed with them. PATOSS, named in question three, is the outside body that independently reviewed it.

  • The Board: Professor John Crawford, University of Aberdeen, a specialist in psychological measurement; Dr John Welch and Dr Clive Skilbeck, consulting clinical neuropsychologists; and Professor Chris Petkov, Newcastle University.
  • Question six has one right answer: no. The assessment is never the only basis for a decision about support: it informs your staff’s judgement, and they decide with what they know of the learner.

See the independent scrutiny behind the assessment

Where we’d be careful

The evidence has limits, and they are worth knowing before you choose any assessment, ours included.

  • The Memory index, at .78 and .68, sits just below the BPS .80 benchmark for high-stakes decisions, and is still within the good range.
  • The studies of who scores differently and of outcomes are doctoral research at Newcastle University: rigorous, independently supervised and being prepared for publication (thesis late 2026 to early 2027), not yet peer-reviewed.

What a flag leads to

A score of 84 or below on an index score starts a needs conversation between the tutor and the learner. From there, a personalised support plan is generated in approximately 10 minutes, for the tutor to tailor, and evidence is captured at the point of delivery, not built retrospectively.

  • So the comparison that matters is the cost of finding and supporting need, not the price of the assessment on its own: a score is where the job starts.
The reasonable adjustments in a learner's support plan in Cognassist: a note that they can be reviewed and updated during one-to-one sessions, then one adjustment for the learner, reword questions if required, tagged Verbal Reasoning.

See how a flag becomes a support decision

See how the support plan works

The ideas, one page each

Keep reading