Funny thing happened to the learning field in the 21st century- numbers now rule the world. Parents, administrators, politicians, clinicians, educators . . . everyone seems to be clamoring for (and clinging to) numerical data. To be sure, scores are important sources of information. But they almost never tell the whole story about a learner.
Qualitative findings are observations made about learner behavior. Such findings may focus on process (how the learner arrived at a response or completed a task) or product (such as accuracy, types/patterns of errors, and organization of work). Quantitative findings are numerical and often normative, meaning that the test developers administered the task to numerous students (usually at different age/grade levels) to generate means and standard scores (like an IQ score). Contrary to what many believe, a standard score does not represent an amount of ability or level of skill. Rather it is a comparison between a learner’s ability level in a defined area or skill with that of other, similar-age students. Both qualitative and quantitative assessment information serve important purposes in assessments. Each type of information has its advantages and disadvantages.
Observations of behavior and nuances of how learners approach tasks help with “reading between the lines” and really understanding their capabilities. Lots of opportunities for dialogue arise when working with learners. During math, for instance, a learner can be asked questions about problems solved incorrectly (like, “How did you get your answer?” “Is this answer right?” “Why is it wrong?” “Where is the mistake?” “Is there a better way to solve it?”). In this clip, I interview Dr. Barrie Morganstein about the virtues of qualitative information:
Work pattern analysis involves analyzing learners’ output to explore what they were thinking as they solved problems and completed tasks. Math, writing, and spelling are well-suited to pattern analysis because a tangible product is generated by the learner (reading, on the other hand, generally does not result in a product).
Quantitative findings are good for comparing a student’s performance to same-age peers while qualitative findings provide critical insight into the learner’s thinking. Using a mix of information means not relying upon any single piece of data, such as a test score. Combining, comparing, and cross-indexing findings deepens the understanding of a learner.
Though many medical and mental disorders are diagnosed with qualitative and quantitative spectrum of data, for decades learning problems have been assessed via a numeric approach that made diagnosis ultimately about subtraction (IQ minus standard achievement score). Public law and regulations facilitated a testing mindset and a gate-keeping mentality regarding eligibility for special education services for students with learning problems. Confidence was eroded in the professional judgment of clinicians such as school psychologists, who had to show numerically that a student had a learning disability, rather than make a case using a portfolio of assessment findings. Now clinicians have used testing so much that it has become synonymous with assessment. Fortunately, the field (and federal law) is shifting towards an approach that looks more closely at the student’s learning needs.
We don’t need for numbers to go away. We just need qualitative findings to go with them in order to tell learners’ stories.
Dr. Craig Pohlman is the Director of Mind Matters at Southeast Psych, a learning program that provides a range of services including assessment, therapy, and professional development. Mind Matters is on Facebook, and can be followed on Twitter @MindMatters_SEP. He is the author of Revealing Minds: Assessing to Understand and Support Struggling Learners.