Without ever having met you, I can tell you something you have in common with your great-great-grandfather.
You have both taken the same kinds of tests. While we have repeatedly innovated content delivery mechanisms, introduced presentation software, learning management systems, and any number of other technological and procedural innovations to disseminate learning content, society has failed to put forward the same intention on assessment innovation. The collateral damage of traditional assessments are well-studied; they overwhelmingly discriminate against newcomers, women, disabled people, Indigenous People, and other equity-deserving groups.
For over 300 years, we have pursued and implement the same assessment modicums with limited improvement in outcomes for under-represented groups. Unsurprisingly, in nearly every case, women of all population segments have faced the greatest harm, discrimination, and barriers to success.
Each new craze proclaims that the house is falling, even as it does nothing to repair the real, foundational problems. Digital white boards that promised to usher in Twenty-First-Century Learning™ now bear ghost-town witness in ten thousand classrooms to the foibles of wasting budgets on flashy, non-transformative technology rather than investing in people and training. MOOCs have proven so hollow that even Udacity has sworn off. Like the flat-earth myth of ‘learning styles,’ Dr John Hattie’s “visible learning” is as academically rigorous as Dr Pepper and as credible as Dr Oz. Even most STEM programs — promising to give learners The Modern Skills They Need® — are as intellectually nutritious as Styrofoam, more focused on improving a school’s testing rank than giving learners meaningful experiences and skills. The common feature all these failed educational panaceas share is a focus on facilitating and verifying the transfer of information. In other words, they’re all built on the same lie.”
Rankin (2020)
Setting the Context
Let’s step back. To appreciate the gravity of the challenge, we must first recognize that assessment is the greatest gatekeeper to achieving career success. Put plainly, if a learner does not pass enough tests, they will not pass a course, earn a degree, nor qualify for most jobs in the market. While employers are increasingly ridding themselves of degree-based requirements for recruitment, these initiatives are still nascent and limited.
In the era of democratized information, the dawn and proliferation of the Internet, and novel tools like ChatGPT, memorization tests remain the single greatest systemic tool employed by institutions of all types (government, educational, regulatory, industry credentialling bodies, etc) to determine whether an individual has earned the right to work and/or grow in their chosen field.
The memorization-driven exam is pervasive in every corner of our societies and economies. In North America, apprenticeship programmes prohibit apprentices from progressing through their apprenticeship if they fail a multiple-choice exam. Women are overwhelmingly underrepresented in the construction trades, even in countries like Germany where apprenticeship and the building trades are relatively sought-after careers.
Traditional assessment models are institution-centered. As opposed to taking learner-centered approaches, our current assessment models are evaluated for their efficiency first and foremost, with learner outcomes an afterthought. This is evidenced multiple times a year when hundreds and thousands of students convene in gymnasiums around the world for their high-stakes final exams. While there exists no scientific basis that this assessment design improves or enhances learner outcomes, we repeat the ritual year after year, semester after semester, perpetuating and propagating gender, ethnic, and ability-based discrimination.
For example, for the first several years when the SAT was offered, boys scored higher than girls on the math section, while girls achieved higher scores on the verbal section. The ETS decided the verbal test needed to be balanced more in favor of boys, and added more questions pertaining to politics, business and sports. No similar efforts were made to balance the math section. “Since then, boys have outscored girls on both the math and verbal sections,” said Zappardino. “So when girls show a superior performance, balancing is required; when boys show the superior performance, no adjustment is necessary.”
(APS, 2020)
Where’s the Discrimination?
The American Physical Society observed that “women and underrepresented minorities typically score significantly lower than men on the standardized tests designed to predict performance.” There are sociological reasons that help explain this. One such reason originates in the way society tends to raise boys versus girls. Boys are rewarded for risk-taking behaviour, where girls are inadvertently raised to be thoughtful and to make sure they aren’t disturbing anyone with their words, thoughts, or actions. As such, the APS remarked that one reason boys tend to perform better on standardized tests are because they feel more empowered to guess than do girls.
Gender bias is evident in many assessment designs. There is significant evidence, for example, that multiple-choice assessments favour males over females (Griselda, 2021a, 2021b). Additionally, assessment design has been seen to intentionally limit female learners. The below example from the Educational Testing Society (ETS) is indicative.
In 2012, long before the arrival of Uber and other similar businesses, the Migration Policy Institute estimated $39 Billion in lost wages – and thus $10 Billion in lost income tax revenue – every year due to college-educated immigrants being mired in lowskilled jobs or unemployment lines in the United States. With immigration rates dramatically higher since 2012, this inequity has persisted and expanded.
Vametric, a world-renowned innovator in talent assessment, helped usher Europe forward in this regard with the transformational overhaul of NHS Care Sector jobs in the mid-2000’s. The introduction of the Care Standards Act required all existing and new workers to meet a competency threshold to help ensure quality of care across the entire system. Through equitable assessments, the entire system flourished, resulting in a ten-fold increase in the number of workers, high levels of worker engagement, and a reduction in time for foreign-trained workers to recertify and engage in work, from an average of two years to as low as 6 weeks.
International research indicates that assessment processes may be influenced by cultural, racial, and gender biases that favour white male apprentices and this further complicates the assessment process for those in the skilled trades from under-represented groups. Research from the United Kingdom has found white male apprentices to be perceived as “good blokes” who may receive informal mentoring and extra help so they will not fail an assessment, while apprentices from equity groups may not receive the same level of support.
(Colley & Jarvis, 2007)
Similar work is being undertaken in the United States, where Vametric’s VALID-8 assessments are so powerful they are overturning state regulatory exams. In one case, all of those appeals came from newcomer women. They had failed the mandated exams, but when their vocational expertise in their equitable assessments were reviewed by the regulator, the learner’s knowledge, skills, and abilities were exceptional and legally defensible, and their exam results were overturned.
What Do We Do From Here?
The future remains uncertain for the global economy, particularly as we think about the talent required to sustain and grow it. In the western world, low birth rates and talent and skills shortages are compounded by the rapidly-retiring workforce, with approximately 30% of all workers set to retire by 2030. In other parts of the world, lower relative wages are compounded by geopolitical risk and/or climate change migration risks. Migration patterns will serve as an overarching theme of the global economy’s talent pool and those countries and employers who get it right will thrive societally and economically.
This means finding a fairer, more transparent, and faster path to legally-defensible, validated assessments of a person’s knowledge, skills, and abilities. Competency-based learning is growing in popularity, and apprenticeship is making a comeback as a learning model that can be applied to nearly any sector. However, learning is only the input. Assessment represents the output, and has an overweight impact on the outcome; that is, poor assessment is resulting in poor labour outcomes.
Competency-based assessment, however, offers a great opportunity for companies, nations, and intermediaries. Rewarding mastery over memorization yields greater results and greater equity by removing gender-based discrimination in traditional assessments. In Canada, companies like Vametric are pairing their assessment approaches with national charitable and
resettlement organizations who are providing additional wraparound supports, such as language, transportation, housing, and career services to address the other systemic barriers that may result in inequities between newcomers and residents, and between men and women.
Assessment is a significantly overlooked lever to greater equity and can be instituted by every organization and employer. The pathway to increasing gender equity and broader equity in the workforce for all can be enhanced with assessments that are transparent, competency-based, and designed in a way that individuals can advocate for their skills and
abilities.
In this model, everyone wins; learners and workers are more empowered and engaged, employers have strong and reliable talent funnels from which to draw highly skilled workers, and governments and training providers can efficiently allocate resources to optimize learner outcomes. It’s already happening; we just need to move faster.