The science behind Cognassist

A guide to the science and development behind Cognassist

Many educators will be familiar with Educational Psychologist reports, but perhaps less so with the process of psychometric testing itself.

Cognassist is an organisation committed to breaking down barriers, not just for learners who require support, but for organisations and individuals looking to become more effective and data-driven in their support provision.

This guide will outline the background of Cognassist’s digital cognitive assessment, as well as comparative paper-based assessments on which our assessment is based.

We will discuss the science and research around cognitive assessments and our basis of validity that gives us the means to ensure, with scientific accuracy, that no learner is left behind.

How do you know the assessment assesses what you want it to assess?

A leading model of human cognition is the Cattell-Horn-Carroll model of cognitive abilities. This model outlines an empirically driven theory of cognition, which was created using multiple meta-analyses of data from various sub-disciplines of psychology such as educational psychology, neuropsychology, cognitive science, neuroscience and clinical psychology.

The tasks included in the Cognassist battery have been mapped onto the CHC model to track the broad reach of the tasks selected and create an evidence-based assessment.

Inter-correlations between tasks are analysed using a standard statistical validation technique called factor analysis. Factor analysis helps to match batteries of cognitive tasks to scientific theory such as CHC. Cognassist tasks load onto expected factors including verbal, memory and visual factors. Correlational analysis, as well as factor analysis has shown that the Cognassist assessment measures cognitive abilities that are related but distinct, as expected from scientific theory.

These individual tasks are well-established within the wider world of psychology and have been used in various controlled settings for many years, developing a large body of evidence for what they measure. For example:

Child development studies – cognitive tasks are used to learn about how children’s minds and academic skills develop and change over time. They are also used to identify potential learning difficulties early on.

Longitudinal studies – seeing how people’s cognition changes over time by using the same tasks, as well as looking at how life events and lifestyles impact cognition over a lifetime.

Work with brain lesion patients – cognitive tasks are used to help diagnose brain injury, but brain injury patients also help us understand human cognition itself when we research which capacities are spared/affected when specific areas of the brain are injured.

Measuring brain activity – looking at brain activity helps researchers understand which cognitive capacities are related to each other and which aren’t, as well as helping us understand which processes are involved in a particular task. As a simple example, if we think a task relies heavily on visual processing, we should see the visual cortex light up in an fMRI machine when we measure the brain activity of somebody completing the task.

Brain pathology research (focusing on Dementia, Parkinson’s and strokes) – cognitive tasks are used to help diagnose pathology but also to understand it. By investigating which cognitive capacities are affected by a disease, we can understand which parts of the brain are being affected. We can also use them to track the progress of a disease.

This helps to give us a very rounded picture of what’s being measured and creates a strong basis of validity grounded in the practical application of research into brain function.

For many learners, understanding how their brain works can be a life-changing revelation that validates a lot of what they’ve experienced previously in education and helps them to learn in a way that works for them.

This requires accuracy and careful processing of data, which would not be possible without our science team and some fantastic consultants who are renown in the field of cognitive research.

Why do you use standard scoring (or normal distribution) to measure people’s abilities in each domain?

Standardised tests are used for multiple purposes in education, including when we measure cognitive abilities. We transform scores into what are called standard scores, which allows us to compare results between different tasks, between people and control for age.

As everyone takes the same assessment through Cognassist, we need to compare each person to others of their age, so we can measure relative performance effectively – it’s not scientifically relevant to compare cognitive scores between 16-year-olds and 64-year-olds.

Standard scoring is a recognised method that allows comparisons of results and is the primary mechanism proposed by both global canons of psychology; the World Health Organisation’s International Classification of Diseases version 11 (ICD-11) and the American Psychological Association’s Diagnostic and Statistical Manual version 5 (DSM-5). No matter the task, standard scores have an average score (or mean) of 100, higher than 100 being above average and lower than 100 being below average.

Transforming scores to standard scores also results in a standard deviation of 15. This is where the threshold of 84 comes from.

A standard score of 84 is essential when identifying a need because this score is just beyond one standard deviation below the mean; a point which is recognised to be scientifically significant and below which an individual can be considered to be at a substantial cognitive disadvantage compared to their peers.

Standard scores conform to what is known as a normal distribution. In a normal distribution, 68% of the population will score somewhere between 85 and 115. About 16% of people will score below 85, and about 16% of people above 115.

Standard deviations are useful, because a score of one standard deviation or more below the mean (84 and below), suggests that that person’s score deviates from the population average more than a significant majority of people.

In a cognitive assessment, a low score will indicate a person having difficulties with this cognitive domain and it is likely they will require support for tasks involved in this area of cognition.

The Joint Council for Qualifications (JCQ) agrees with this scoring.

The Access Arrangements and Reasonable Adjustments regulations and learner support criteria state:

“So as not to give an unfair advantage, the assessor’s report (Part 2 of Form 8) must confirm that the candidate has: at least one below average standardised score of 84 or less which relates to an assessment of:

• speed of reading (see paragraph 7.5.10); or
• speed of writing (see paragraph 7.5.11); or
• cognitive processing measures which have a substantial and long term adverse effect on speed of working (see paragraph 7.5.12).”

To top it off, a learning need is also defined within the Education Act 1996 as:
“the person has a significantly greater difficulty in learning than the majority of persons of the same age[.]”

Another reason why we use the age-standardised score to maintain validity and meet established criteria.

Can a learner access support if they identify in only one domain?

The Cognassist neurodiversity assessment has been designed to measure a broad spectrum of cognitive abilities, rather than a single, combined measure. These nine domains can and should be measured separately. Therefore, support should be given if a learner identifies in any of the domains. We can break this down further.

Within the CHC model, it is acknowledged that many of the domains of cognition are related, and often work together as we perform tasks, but still represent different abilities, which should be measured separately if we are to build a more comprehensive understanding of a person’s brain:

“[D]ifferent [cognitive] abilities do not reflect completely independent (uncorrelated or orthogonal) traits. However, they can, as is evident from the vast body of literature that supports their existence, be reliably distinguished from one another and therefore represent unique, albeit related, abilities (see Keith & Reynolds, 2012).” (Flanagan, D. P. & Dixon, S. G., 2014)

From statistical analysis of our own dataset, we calculated that the intercorrelations between each task are low to moderate (0.1-0.4). Some correlation is a good thing because it reflects our knowledge of how the brain works, and its inherently connected nature; however, lower correlations also show that each task is measuring a unique aspect of cognition.

What this is all building to say is that, if we know we can measure each domain separately and that they are indeed measuring different abilities, then we cannot withhold vital support when someone identifies with a need in one of these areas.

A lower score in a domain can represent significant challenges that often affects people’s performance in a specific range of tasks. Ignoring a low score in one domain means this learner will continue to struggle with this aspect of learning and thinking because they are not as effective in this type of information processing. They will also not benefit from strategies that could help them to rely on their strengths and mitigate the negative effects of areas where they struggle.

How are strategies planned?

1. Planning the curriculum
   Within the standards framework, we identify common cross-disciplinary skills and behaviours at each level of study within further and higher education. For example, if a learner is studying at level 4 or above, we might focus on strategies that reflect certain managerial skills and admin tasks. We also focus on soft skills and effective study strategies to help learners better understand and implement skills that would be expected at their level of study but not necessarily taught directly as part of the core curriculum. For example, effective leadership and delegation skills.
   For learners who match on our assessment, these soft skills and tasks may not be as intuitive as they are for others; providing them with step by step strategies helps to crystalise this information and build their confidence, which can have a direct impact on their likelihood of completion.
2. Consultation with experts
   Our consulting neuropsychologists look at each strategy to define a level of difficulty, based on the different cognitive domains. They also identify which strategies will be the most beneficial for each learner, addressing the areas of learning they find most difficult to provide targeted interventions that give learners the best possible chance of success.
   For example, a strategy on planning and organising work effectively is likely to be more relevant for learners who require support with executive function; however, it may also be relevant for learners who struggle with numeracy, which may be less obvious but no less adverse if not addressed. The relevance and difficulty of tasks is not always obvious, which is why we rely on scientists who have spent years studying the brain and how we think and learn. This step also informs how we design and write each strategy to better suit each learner.
3. Design
   Our learning design team take over from here. They build each strategy into the platform using state of the art authoring tools with world-class credentials, which allows us to integrate multimedia options to support learning and present information in different ways to increase accessibility.
4. Content customisation
   We use focus groups at each level of study to ensure our content is useful to learners and presented in an easy-to-understand way, tailored to each of the domains so learners are getting the most out of the content available.
5. Enhancing content
   We regularly audit our learning strategy content to ensure it is still fit for purpose and reflects the demands of a learner’s chosen level of study. We also rely on learner feedback, built into each strategy module, to assess the quality and effectiveness of our content for those it is there to support.

Accessibility

Our mission is to make learning more accessible and improve the lives of learners who might otherwise struggle in education. We are constantly evaluating and refining our platform to drive this accessibility. We pride ourselves on keeping to high standards of both science and design to build a platform that’s intuitive and invaluable to learners and staff.

Our platform is measured against the Web Content Accessibility Guidelines (WCAG 2.1), an international metric for web inclusivity, and our internal content and design guidelines have been structured around these best practice recommendations. As these recommendations are updated, we continue to iterate our platform to meet the principles of accessibility, usability and inclusion for all, using the best available knowledge.

It is the same for the science that fuels our assessment.

Modern cognitive science has only been around for about 140 years. As technology rapidly improves, our ability to measure and understand the brain increases. The human mind is still relatively unknown, we have no idea where consciousness itself comes from or how we develop our sense of self throughout our lives.

However, you can still provide valuable support to learners without solving the hard problem of consciousness.

What is clear to the scientific community and many educators who’ve used Cognassist, is that cognition plays a vital role in learning. It is the reason we are able to learn anything at all. Surely that’s worth knowing more about? To make learning more flexible and help tutors effectively identify learners who require more support early on in their learning journey.

If you’re still looking for more, please take a look at our reference table below for an overview of specific rules we use to write, build and design each strategy based on these eight domains.

How do you know these strategies are effective?

As we mentioned, we are constantly gathering feedback from learners using Cognassist.

Each strategy has a feedback survey afterwards, and from 173,882 learner responses, we have found that 89% would apply the skill they learned to their work and study and 98% of the same number found their strategy useful.

The National Achievement Rate Tables (NARTs) shows the achievement rates of disabled learners or learners who experience learning difficulties (LDD).

Data from these tables shows that organisations that have used Cognassist for over two years have significantly higher than average achievement rate for their LDD learners – this is while having a larger than average cohort of LDD learners.

Even though they have more LDD learners, they still achieve higher than the national average, which goes to show that with the right support, more people are capable of achieving their educational goals.

And here are the numbers:

Data shows that LDD learners of Cognassist clients who have embedded the Cognassist assessment and support strategy tool into their processes had an average achievement rate of 70%, compared to the national average of 64.6% (an 8.36 percentage difference).

We’ve helped many clients increase learner attainment, like Bradford College, who increased LDD learner attainment by 10% in the NARTs in one year.

Some of our clients have achieved 100% achievement, so there is huge possibility for your learners to succeed and unlock their learning potential.

Hopefully, this gives you further insight into our platform, the assessment and learning support modules that have all been carefully designed to bring out the best learning experience and improve the lives of learners one step at a time.